Country or region and language

    Choose your country or region and your language.

    Suggested for you

      All countries and regions

        Choose your character

        It shows in the top bar and on your profile. Change it any time.

        Projects/Pennant/Case study

        Is the annual offer result real?

        A checkout test offered half the visitors an annual plan. The dashboard says it worked; count the test fairly and decide whether the difference is real before it ships.

        Case studyPennant · fictional company1 h3 stagesSQL → pandas → Recommendation

        You write the recommendationIt goes to the product lead. Say what the evidence shows and what still needs checking.

        Pennant

        Fitness app

        About the company

        A fitness app with guided workouts, sold as a monthly or annual subscription in the UK.

        Pennant gives people guided workouts and training plans they can follow at home or in the gym. Subscribers pay by the month or for a year up front, and they sign up through the app stores, a friend's referral or a paid social ad. They live all over the UK, including London, the Midlands, Scotland and Wales. About one in four new monthly subscribers cancels within three months. You work in the data team with two groups: retention, which tries to keep new subscribers training, and product, which runs tests in the checkout.

        More at Pennant

        Founded
        2019
        Head office
        Manchester, UK
        Plans
        Monthly or annual
        Employees
        About 70

        Who writes the briefs

        • Hannah OkoroHead of Retention
        • Tom AshbyProduct Lead

        A fictional company, invented for practice. Meet the other companies

        The situation

        From
        Tom AshbyProduct Lead
        To
        Data team
        Subject
        Annual offer test, ship it?

        For two weeks in September, half the people reaching checkout saw an offer to start on the annual plan. The dashboard shows more upgrades in the offer group and I would like to ship it to everyone on Monday. Can you check the numbers hold up first?

        You have the assignment log and the upgrades. Visitors who switched between phone and tablet were sometimes assigned twice, and a few people upgraded before the test began.

        Done looks like: both groups counted fairly, a significance test on the difference, and a clear recommendation: ship, extend the test, or drop it.

        • Data: about 1,900 visitors assigned over 14 days, and their upgrades.
        • You deliver a fair count of the test, a z-test and a recommendation.
        • The recommendation is reviewed against a four-point rubric.

        The evidence

        SQL

        1. Count the test fairly

        Each visitor once, in their first group, and only upgrades the offer could have caused.

        Reads:
        assignments · upgrades
        Produces:
        test_results
        Time:
        20 minutes
        Open SQLOne Course in SQL or All Access
        pandas

        2. Is the difference real?

        A two-proportion z-test on the upgrade rates.

        Reads:
        test_results
        Produces:
        Time:
        20 minutes
        Open pandasOne Course in pandas or All Access

        3. Your recommendation

        Use the reference evidence or your checked tables. Make the call, support it with a comparison, and say what would change your mind. Allow 20 minutes.

        Case study access

        Free

        $0

        Read all briefs and paths. Tool stages and written work are locked.

        Keep browsing

        One Course

        $17.99/month

        Practice the stages in your chosen tool. Reference tables let you start at that stage.

        All Access

        $29.99/month

        Every stage in every format, plus optional spoken readouts using dumbbells.

        Your Work Gym account

        Save your progress and pick up on another device. Your browser's progress comes with you.

        or

        You can keep practicing without an account.

        Help and feedback

        Ask a question, tell us what is not working, or share an idea. We reply by email.

        What is it about?

        We use it only to reply to you.