Country or region and language

    Choose your country or region and your language.

    Suggested for you

      All countries and regions

        Choose your character

        It shows in the top bar and on your profile. Change it any time.

        Building with language models course

        Held-out identity boundaries

        Opening Building with language models
        Your turn

        LLM evaluation and held-out cases

        Development and evaluation

        A case used to change a prompt is no longer independent evidence of how it generalizes. Keep development and held-out case identities disjoint. For a fictional policy assistant, reserve a case with a source revision absent from the development examples; check the split by identity before scoring any result.

        Freeze the cases and versions used in a comparison. Report missing outputs and newly failed cases rather than quietly dropping them from the denominator. A candidate with a better average still fails release if it breaks a mandatory tenant boundary.

        Local evaluation harness

        The course grades prepared outputs with explicit Python rules. It teaches split integrity, denominators and release gates, not the measured quality of a live LLM. Continue with the grading rubric and regression comparison.

        Product setup changes. The OpenAI Evals documentation, checked November 14, 2026, marks its hosted Evals surface deprecated and read-only, with closure scheduled for November 30. This lesson does not ask you to create a new hosted Evals job. Use current vendor documentation when moving the method into a live environment.

        Your Work Gym account

        Save your progress and pick up on another device. Your browser's progress comes with you.

        or

        You can keep practicing without an account.

        Change your plan

        Checking the price

        Help and feedback

        Ask a question, tell us what is not working, or share an idea. We reply by email.

        What is it about?

        We use it only to reply to you.