Easier to read with cream paper, a tint, more contrast or larger text? Choose your colours and text, once, for every page.

Free module

Evaluating AI systems in depth

Learn to evaluate AI systems in depth: golden sets, regression tests, judge pitfalls, hallucination rate, retrieval metrics and synthetic eval data.

6 lessons410 min lessonSoftware engineering

The lessons

Each lesson is free to read in full. Create a free account to run its commands in a live lab and to sit its quizzes and tasks.

  1. 1 · Container lab

    Is your evaluation set good enough?

    I can say whether an evaluation set is good enough to trust.

    Read the lesson

  2. 2 · Container lab

    Make a worse prompt fail the build

    I can make a worse prompt fail a build.

    Read the lesson

  3. 3 · Container lab

    How a judge can be biased

    I can find out how a grader is biased before I rely on it.

    Read the lesson

  4. 4 · Container lab

    Measuring a hallucination rate

    I can measure how often an answer says what its source does not.

    Read the lesson

  5. 5 · Container lab

    Scoring search apart from the answer

    I can score the search apart from the answer.

    Read the lesson

  6. 6 · Container lab

    Making synthetic evaluation cases

    I can make new cases from a seed and say they are not copies.

    Read the lesson

Worked in a real lab

Every lesson of this module is worked in a lab started for you alone. The labs it uses:

Teaching a class? Set this module for your students with a teacher guide, marked assignments and a lab for every student.