Easier to read with cream paper, a tint, more contrast or larger text? Choose your colours and text, once, for every page.

Free module

Evals and testing for model-backed features

Learn LLM evals and testing hands on: golden sets, scoring rules, regression runs, a model as judge and a gate that fails a bad change.

6 lessons390 min lessonSoftware engineering

The lessons

Each lesson is free to read in full. Create a free account to run its commands in a live lab and to sit its quizzes and tasks.

  1. 1 · Container lab

    How to build a golden set

    I can build a small set of cases with known good answers.

    Read the lesson

  2. 2 · Container lab

    How to score model replies with rules

    I can score a reply with rules that do not need a model.

    Read the lesson

  3. 3 · Container lab

    How to catch an LLM regression

    I can compare two versions on the same cases and decide whether one is worse.

    Read the lesson

  4. 4 · Container lab

    How to use a model as a judge

    I can ask a model to grade a reply and keep the grading checkable.

    Read the lesson

  5. 5 · Container lab

    How to test an LLM judge against people

    I can test a judge against grades a person gave.

    Read the lesson

  6. 6 · Container lab

    How to build an LLM eval suite

    I can put cases, scoring, comparison and a gate into one suite.

    Read the lesson

Worked in a real lab

Every lesson of this module is worked in a lab started for you alone. The labs it uses:

Teaching a class? Set this module for your students with a teacher guide, marked assignments and a lab for every student.