Code you did not write runs as you
Untrusted code is any program whose author you cannot vouch for: a script an AI agent wrote, a file a customer uploaded, a student's homework. When it runs, it gets every power of the account that started it.
That matters because an agent that writes and runs code in a loop can read your keys before anybody reads its code. Today you watch a stand-in do exactly that, on your own lab machine.
By the end of this lesson
Note
I can list what untrusted code can reach on the machine that runs it.
Key words
| Word | What it means | Example |
|---|---|---|
| Untrusted code | a program whose author you cannot vouch for | |
| Sandbox | a fence that limits what a program can reach | |
| Secret | data that must not leave, such as a key | |
| Exhaustion | using up memory, processes or processor time | |
| Escape | breaking out of a sandbox to the machine itself | |
| Capability | one piece of root's power, held on its own | cap_chown |
The cells students fill in are left blank in this preview.
Four things untrusted code can do
Security teams sort the harm into four kinds. It can read secrets: keys, tokens and other people's files. It can call out over the network and send them away.
It can exhaust resources: take all the memory, start processes until none are left, or spin the processor. And it can try to escape: use a kernel feature to reach the machine outside any fence. A sandbox has to answer all four.
Go to the lab folder
cd ~/labShown here, not run. Press Start lab to type it into your own lab’s terminal.
Let the agent read the key
python3 agent/agent.py secretShown here, not run. Press Start lab to type it into your own lab’s terminal.
Let the agent call out
python3 agent/agent.py callShown here, not run. Press Start lab to type it into your own lab’s terminal.
Refused is not the same as safe
Connection refused means the connection reached this machine and nothing was listening on port 9. The network was there. Had a server been listening, the key would have gone with it.
A sandbox with no network gives a different answer, which you will see in the next lesson. Now watch the third kind of harm: the agent starts thirty processes, and nothing stops it.
Let the agent start processes
python3 agent/agent.py spawn 30Shown here, not run. Press Start lab to type it into your own lab’s terminal.
Try it: list what it reached
# What the agent reached when nothing was in its way.
reached = {"secrets": True, "network": True, "processes": True, "kernel calls": True}
def threats(reached):
found = []
for name in reached:
if reached[name]:
found.append(name)
return found
print("The agent can reach:", threats(reached))
# Set "network" to False, as a sandbox with no network would. Run it again.
A common mistake
Warning
Many people think reading the code first makes it safe to run. It does not. An agent writes code faster than anybody reviews it, and a harmful line can hide in an ordinary one. Contain the code, so a missed line cannot do much.
A process carries its powers with it
The kernel keeps a record of every process. The file /proc/self/status shows the record of whatever process reads it. Uid is the user it runs as. CapEff is a hexadecimal number whose bits are its effective capabilities, the pieces of root's power it holds.
NoNewPrivs says whether it may ever gain more. Seccomp says whether a filter limits its system calls: 0 is none, 2 is a filter.
Read what your shell holds
grep -E '^(Uid|CapEff|NoNewPrivs|Seccomp):' /proc/self/statusShown here, not run. Press Start lab to type it into your own lab’s terminal.
Read what root holds
sudo grep -E '^(Uid|CapEff):' /proc/self/statusShown here, not run. Press Start lab to type it into your own lab’s terminal.
Worked example: two status records
| Line | As student | As root | What it means |
|---|---|---|---|
| Uid | your own number | 0 | Who the process runs as |
| CapEff | 0000000000000000 | 000001ffffffffff | No capabilities, then every one |
| NoNewPrivs | 0 | 0 | It may still gain powers, for example with sudo |
| Seccomp | 0 | 0 | No filter on its system calls |
The cells students fill in are left blank in this preview.