- From
- Tom AshbyProduct Lead
- To
- Data team
- Subject
- Annual offer test, ship it?
For two weeks in September, half the people reaching checkout saw an offer to start on the annual plan. The dashboard shows more upgrades in the offer group and I would like to ship it to everyone on Monday. Can you check the numbers hold up first?
You have the assignment log and the upgrades. Visitors who switched between phone and tablet were sometimes assigned twice, and a few people upgraded before the test began.
Done looks like: both groups counted fairly, a significance test on the difference, and a clear recommendation: ship, extend the test, or drop it.