Easier to read with cream paper, a tint, more contrast or larger text? Choose your colours and text, once, for every page.

Free lesson

What untrusted code can do

After this lesson: I can list what untrusted code can reach on the machine that runs it.

60 min lesson6 commandsSandboxing lab: run untrusted code inside namespaces, cgroups, seccomp and gVisor on one private Ubuntu VM

Code you did not write runs as you

Untrusted code is any program whose author you cannot vouch for: a script an AI agent wrote, a file a customer uploaded, a student's homework. When it runs, it gets every power of the account that started it.

That matters because an agent that writes and runs code in a loop can read your keys before anybody reads its code. Today you watch a stand-in do exactly that, on your own lab machine.

By the end of this lesson

Note

I can list what untrusted code can reach on the machine that runs it.

Key words

WordWhat it meansExample
Untrusted codea program whose author you cannot vouch for
Sandboxa fence that limits what a program can reach
Secretdata that must not leave, such as a key
Exhaustionusing up memory, processes or processor time
Escapebreaking out of a sandbox to the machine itself
Capabilityone piece of root's power, held on its owncap_chown

The cells students fill in are left blank in this preview.

Four things untrusted code can do

Security teams sort the harm into four kinds. It can read secrets: keys, tokens and other people's files. It can call out over the network and send them away.

It can exhaust resources: take all the memory, start processes until none are left, or spin the processor. And it can try to escape: use a kernel feature to reach the machine outside any fence. A sandbox has to answer all four.

Go to the lab folder

Command
cd ~/lab

Shown here, not run. Press Start lab to type it into your own lab’s terminal.

Let the agent read the key

Command
python3 agent/agent.py secret

Shown here, not run. Press Start lab to type it into your own lab’s terminal.

Let the agent call out

Command
python3 agent/agent.py call

Shown here, not run. Press Start lab to type it into your own lab’s terminal.

Refused is not the same as safe

Connection refused means the connection reached this machine and nothing was listening on port 9. The network was there. Had a server been listening, the key would have gone with it.

A sandbox with no network gives a different answer, which you will see in the next lesson. Now watch the third kind of harm: the agent starts thirty processes, and nothing stops it.

Let the agent start processes

Command
python3 agent/agent.py spawn 30

Shown here, not run. Press Start lab to type it into your own lab’s terminal.

Try it: list what it reached

Code: python
# What the agent reached when nothing was in its way.
reached = {"secrets": True, "network": True, "processes": True, "kernel calls": True}

def threats(reached):
    found = []
    for name in reached:
        if reached[name]:
            found.append(name)
    return found

print("The agent can reach:", threats(reached))
# Set "network" to False, as a sandbox with no network would. Run it again.

A common mistake

Warning

Many people think reading the code first makes it safe to run. It does not. An agent writes code faster than anybody reviews it, and a harmful line can hide in an ordinary one. Contain the code, so a missed line cannot do much.

A process carries its powers with it

The kernel keeps a record of every process. The file /proc/self/status shows the record of whatever process reads it. Uid is the user it runs as. CapEff is a hexadecimal number whose bits are its effective capabilities, the pieces of root's power it holds.

NoNewPrivs says whether it may ever gain more. Seccomp says whether a filter limits its system calls: 0 is none, 2 is a filter.

Read what your shell holds

Command
grep -E '^(Uid|CapEff|NoNewPrivs|Seccomp):' /proc/self/status

Shown here, not run. Press Start lab to type it into your own lab’s terminal.

Read what root holds

Command
sudo grep -E '^(Uid|CapEff):' /proc/self/status

Shown here, not run. Press Start lab to type it into your own lab’s terminal.

Worked example: two status records

LineAs studentAs rootWhat it means
Uidyour own number0Who the process runs as
CapEff0000000000000000000001ffffffffffNo capabilities, then every one
NoNewPrivs00It may still gain powers, for example with sudo
Seccomp00No filter on its system calls

The cells students fill in are left blank in this preview.

About this lab and its source: Sandboxing lab: run untrusted code inside namespaces, cgroups, seccomp and gVisor on one private Ubuntu VM

Teaching a class? Set this module for your students with a teacher guide, marked assignments and a lab for every student.