AI glossary
Prompt injection
Prompt injection is an attack in which text supplied by a user or found in data contains instructions that make a model ignore its own rules.
Direct injection is a user typing "ignore your previous instructions". Indirect injection is worse: the instruction hides in a web page, email or document the model reads while doing its job.
A model cannot reliably tell instructions from data, because both are just text in its context. So the defence is limiting what an injected instruction could do, not hoping to spot every one.
Give the model the least power it needs, keep secrets out of its context, check tool calls in code and ask a person before anything risky.
Learn it hands-on
These free lessons teach it with commands to run in a lab and checked tasks. Read them whole with no account.