AI glossary
Caching
Caching keeps the result of an earlier model call or computation so the same request can be answered again without paying for it twice.
The simplest cache keys on the exact prompt and settings and stores the reply. A repeated question then costs nothing and returns at once.
Many providers also cache a long, unchanging start of a prompt, such as a system prompt or a document, which makes later calls cheaper and faster.
A cache must know when to forget. Include the model version and prompt version in the key, so a change does not serve yesterday's answers.
Learn it hands-on
These free lessons teach it with commands to run in a lab and checked tasks. Read them whole with no account.