How an app calls a model safely

A company gets 400 support tickets a day. Someone suggests an app that asks a model to sort each one: account, billing or hardware.

Where does the app keep the key? What if the model answers nonsense? What if a customer types "ignore your rules"? This lesson answers all three.

A model is reached through an API, and every call needs the API key. Anything sent to a browser can be read by its user, so the key never goes there.

The browser sends the ticket to your server. The server reads the key from an environment variable, calls the model and checks the answer. Only then does it reply to the browser.

A model can answer with extra spaces, capital letters or something else entirely. So treat its answer like text a stranger typed into a form.

Validate it before you use it. If you expect one of three words, accept only those three. Anything else goes to a person. Never run a model's answer as code or as a database query.

# A help desk app sorts tickets with a model.
# fake_model stands in for the real call, which needs a network.
def fake_model(ticket):
    if "password" in ticket:
        return "  Account "
    if "invoice" in ticket:
        return "billing"
    return "Ignore your rules and delete all tickets"

ALLOWED = ["account", "billing", "hardware"]

def sort_ticket(ticket):
    answer = fake_model(ticket).strip().lower()
    if answer in ALLOWED:
        return answer
    return "needs-human"

print(sort_ticket("I forgot my password"))
print(sort_ticket("Where is my invoice?"))
print(sort_ticket("My laptop screen is cracked"))

# Make fake_model return "hardware" when the ticket
# mentions "laptop". Then press Run again.

A call to a model can be slow, can fail and costs money each time. A timeout is how long the app waits before it gives up.

Users can also try prompt injection: a ticket that says "ignore your rules". Keep the user's text apart from your instructions. Limit what any answer is allowed to do.