Easier to read with cream paper, a tint, more contrast or larger text? Choose your colours and text, once, for every page.

Free module

Evaluating agents

Learn evaluating AI agents hands on: score their steps, check the state they leave, compare runs fairly, test dialogues and gate a change.

6 lessons400 min lessonSoftware engineering

The lessons

Each lesson is free to read in full. Create a free account to run its commands in a live lab and to sit its quizzes and tasks.

  1. 1 · Container lab

    How to evaluate an AI agent trajectory

    I can score the steps an agent took against a reference path.

    Read the lesson

  2. 2 · Container lab

    How to check what an AI agent changed

    I can check what a run changed instead of what it said.

    Read the lesson

  3. 3 · Container lab

    Is my new AI agent really better?

    I can say whether one agent is better than another when each run differs.

    Read the lesson

  4. 4 · Container lab

    How to test a chatbot with simulated users

    I can test a dialogue with a scripted user and score the conversation.

    Read the lesson

  5. 5 · Container lab

    How to build eval cases from production traces

    I can turn a failure in use into a case without keeping what should not be kept.

    Read the lesson

  6. 6 · Container lab

    How to build an AI agent eval suite

    I can put trajectory, state and repeated runs into one gate.

    Read the lesson

Worked in a real lab

Every lesson of this module is worked in a lab started for you alone. The labs it uses:

Teaching a class? Set this module for your students with a teacher guide, marked assignments and a lab for every student.