AI glossary
Eval
An eval is a repeatable test of an AI feature: a fixed set of inputs, a way to score each output, and a result you can compare between versions.
Model outputs vary, so a single exact-match assertion rarely works. An eval runs many cases and reports a score, such as 47 of 50 summaries containing the order number.
Scores come from rules in code, from comparing with a reference answer, or from another model acting as a judge. Cheap code checks first, judges where code cannot decide.
Run evals on every change to a prompt, a model or retrieval, the way you run unit tests. A change that feels better in three tries can be worse over fifty.
Learn it hands-on
These free lessons teach it with commands to run in a lab and checked tasks. Read them whole with no account.