AI glossary
Red teaming
Red teaming is deliberately attacking your own AI system, trying to make it misbehave, leak data or misuse its tools, so you find weaknesses before others do.
A red team writes hostile inputs: injection hidden in documents, requests that walk round the rules step by step, tool calls aimed outside their bounds.
Each attack that works becomes a test case, so a later change that reopens the hole is caught.
Red teaming finds problems; it does not prove there are none. Pair it with limits in code so the attacks you did not think of are bounded too.
Learn it hands-on
These free lessons teach it with commands to run in a lab and checked tasks. Read them whole with no account.