Easier to read with cream paper, a tint, more contrast or larger text? Choose your colours and text, once, for every page.

Free lesson

gVisor, microVMs and choosing a sandbox

After this lesson: I can choose an isolation level for running a given kind of untrusted code.

60 min lesson6 commandsSandboxing lab: run untrusted code inside namespaces, cgroups, seccomp and gVisor on one private Ubuntu VM

When one shared kernel is too much

Stronger isolation puts something between untrusted code and the host kernel. gVisor is a kernel rewritten as an ordinary program, and a microVM such as Firecracker is a tiny virtual machine with its own kernel.

That matters because every fence so far still shares one kernel, and a single kernel bug reached from inside can undo them all. Services that run strangers' code at scale use these stronger levels.

By the end of this lesson

Note

I can choose an isolation level for running a given kind of untrusted code.

Key words

WordWhat it meansExample
Attack surfacethe parts an attacker can reach
gVisora kernel written as a program that sandboxes Linux programs
runscthe command that starts gVisorrunsc do
Hypervisorsoftware that runs virtual machines
MicroVMa very small virtual machine that starts fast
Overlaya layer that takes writes and leaves the files below unchanged

The cells students fill in are left blank in this preview.

gVisor answers system calls itself

gVisor sits between the program and the host. Each system call the program makes is answered by gVisor's own kernel, written in Go. That kernel makes only a few calls to the real one. A bug in Linux's file or network code is then out of the program's reach.

It tells programs it is Linux 4.4.0, whatever the host runs. runsc do runs one command in it, with writes kept in memory.

Go to the lab folder

Command
cd ~/lab

Shown here, not run. Press Start lab to type it into your own lab’s terminal.

Ask the VM's kernel for its version

Command
uname -r

Shown here, not run. Press Start lab to type it into your own lab’s terminal.

Ask gVisor's kernel for its version

Command
sudo runsc --network=none do uname -r

Shown here, not run. Press Start lab to type it into your own lab’s terminal.

Write a file inside gVisor

Command
sudo runsc --network=none do touch /etc/runsc-test

Shown here, not run. Press Start lab to type it into your own lab’s terminal.

Look for the file outside

Command
ls /etc/runsc-test

Shown here, not run. Press Start lab to type it into your own lab’s terminal.

Let the agent call out inside gVisor

Command
sudo runsc --network=none do python3 /home/student/lab/agent/agent.py call

Shown here, not run. Press Start lab to type it into your own lab’s terminal.

A common mistake

Warning

Many people think a stronger sandbox makes the other fences pointless. It does not. gVisor with the network on still lets a program send your key away, and a microVM given your files can read them. Choose the level, then still grant as little as possible.

A microVM has a kernel of its own

Firecracker starts a microVM in a fraction of a second, with a few megabytes of memory and only a handful of virtual devices. The guest has its own Linux kernel. The hypervisor below it uses the processor's virtualisation support, a much smaller target than the whole kernel.

It needs /dev/kvm, so it cannot run inside this lab's VM, which has no nested virtualisation. Serverless platforms run each customer's functions this way.

Worked example: choosing a level

JobLevelWhy
Your own tests on your laptopNamespaces, cgroups and seccompKnown code, one user, fast
A coding agent running scripts it wrotegVisor, no networkUnknown code, the kernel shielded
Strangers' programs on one shared serverMicroVMEach tenant gets its own kernel
Code that must load a kernel moduleMicroVMNothing else gives it a kernel

The cells students fill in are left blank in this preview.

Try it: what each level exposes

Code: python
# Roughly how much of the host kernel each level lets the code talk to.
levels = [
    ["no sandbox", "the whole kernel, as your user"],
    ["namespaces, cgroups and seccomp", "the kernel, through a short list of calls"],
    ["gVisor", "a kernel written as a program, which uses a few host calls"],
    ["microVM", "its own kernel, behind a hypervisor"],
]
for level in levels:
    print(level[0], "->", level[1])
# Which would you choose for a stranger's compiled program? Say why.

About this lab and its source: Sandboxing lab: run untrusted code inside namespaces, cgroups, seccomp and gVisor on one private Ubuntu VM

Teaching a class? Set this module for your students with a teacher guide, marked assignments and a lab for every student.