AI glossary
Canary rollout
A canary rollout sends a small share of real traffic to a new model or prompt first, and widens it only if quality and errors stay within agreed limits.
Five per cent of requests go to the new version while the rest stay on the old one. You compare error rates, cost, latency and quality signals between the two.
Decide in advance what counts as a failure and what happens then, so rolling back is a switch, not a meeting.
A canary catches what offline evals missed, such as real users phrasing things differently. It is a second check, not a replacement for evals.
Learn it hands-on
These free lessons teach it with commands to run in a lab and checked tasks. Read them whole with no account.