Easier to read with cream paper, a tint, more contrast or larger text? Choose your colours and text, once, for every page.

Free module

Cost, tracing and reliability

Learn cost, tracing and reliability for LLM features hands on: spans, caching, budgets, latency limits, fallbacks and model routing.

6 lessons375 min lessonSoftware engineering

The lessons

Each lesson is free to read in full. Create a free account to run its commands in a live lab and to sit its quizzes and tasks.

  1. 1 · Container lab

    How to trace an LLM call

    I can record what each call did, so that I can find where a failure or a delay came from.

    Read the lesson

  2. 2 · Container lab

    How to cache LLM responses

    I can cache model answers safely and say exactly what makes a hit.

    Read the lesson

  3. 3 · Container lab

    LLM token cost and budget limits

    I can stop a feature spending more than it should, with a per-user limit, a daily limit and a clear message.

    Read the lesson

  4. 4 · Container lab

    LLM latency, timeouts and percentiles

    I can say where the time goes in a model call and set a timeout from recorded durations.

    Read the lesson

  5. 5 · Container lab

    LLM fallbacks when a call fails

    I can keep a feature working when a model call fails by choosing a fallback for each kind of failure.

    Read the lesson

  6. 6 · Container lab

    How to route requests between LLMs

    I can route each request to the cheapest model that is good enough and check the choice against recorded cases.

    Read the lesson

Worked in a real lab

Every lesson of this module is worked in a lab started for you alone. The labs it uses:

Teaching a class? Set this module for your students with a teacher guide, marked assignments and a lab for every student.