AI glossary
Model routing
Model routing sends each request to the model best suited to it, often a small cheap model for easy requests and a larger one only when needed.
A router can be a simple rule, such as short questions to the small model, or a classifier that predicts how hard a request is.
Routing goes with fallbacks: if a model times out or is refused, the request moves to another one so the feature keeps working.
Every route needs its own evals. A cheaper model that fails quietly on one kind of request saves money and costs trust.
Learn it hands-on
These free lessons teach it with commands to run in a lab and checked tasks. Read them whole with no account.