A limit the program cannot argue with
A control group, or cgroup, is a set of processes that the Linux kernel counts and limits together. It limits their memory, their processor time and how many processes they start. Version 2 is the one current systems use.
That matters because untrusted code that grabs all the memory takes the machine down for everyone. A cgroup stops it, and today you watch the kernel do it.
By the end of this lesson
Note
I can set memory, CPU and process limits on a program with cgroups.
Key words
| Word | What it means | Example |
|---|---|---|
| Control group | processes the kernel counts and limits together | |
| memory.max | the most memory the group may hold, in bytes | 67108864 |
| cpu.max | processor time allowed in each period | 20000 100000 |
| pids.max | the most processes the group may hold | 20 |
| OOM kill | the kernel killing a process that is out of memory | |
| Fork bomb | a program that starts processes without end |
The cells students fill in are left blank in this preview.
A cgroup is a folder of files
Every cgroup is a folder under /sys/fs/cgroup. Making a folder makes a group. Writing a number into one of its files sets a limit, and writing a process number into cgroup.procs moves that process in. Every child it starts joins the same group.
The lab's helper, sbx limit, does exactly that and nothing more. With --show it prints each file it writes. Read /usr/local/bin/sbx to check.
Go to the lab folder
cd ~/labShown here, not run. Press Start lab to type it into your own lab’s terminal.
See the files sbx limit writes
sudo sbx limit --memory 64M --cpu 20 --pids 20 --show -- trueShown here, not run. Press Start lab to type it into your own lab’s terminal.
Worked example: three limits as files
| Option | File | What sbx wrote | Why that number |
|---|---|---|---|
| --memory 64M | memory.max | 67108864 | 64 times 1024 times 1024 bytes |
| --cpu 20 | cpu.max | 20000 100000 | 20 ms of every 100 ms |
| --pids 20 | pids.max | 20 | At most 20 processes and threads |
The cells students fill in are left blank in this preview.
Over the memory limit, the kernel kills
A program inside a group can ask for more memory than memory.max allows. The kernel first tries to reclaim memory, then kills a process in the group. The kill is counted in the file memory.events, on the line oom_kill.
The agent below asks for 200 MB, 10 MB at a time, inside a 64 MB limit. Predict what it prints before you press Run.
Ask for 200 MB inside 64 MB
sudo sbx limit --memory 64M -- python3 agent/agent.py memory 200Shown here, not run. Press Start lab to type it into your own lab’s terminal.
Ask for 200 MB with no limit
python3 agent/agent.py memory 200Shown here, not run. Press Start lab to type it into your own lab’s terminal.
A common mistake
Warning
Many people think a program over its memory limit gets an error it can handle. On Linux it usually does not. The kernel kills it with a signal it cannot catch, so the program never prints its last line. Exit status 137 is that kill.
A process limit stops a fork bomb
A fork bomb starts processes until the machine has no room for one more, and then nothing can start, not even your shell. pids.max counts every process and thread in the group.
When the group is full, the next fork fails with Resource temporarily unavailable, and pids.events counts it on its max line. The program carries on with what it has.
Start 30 processes inside 20
sudo sbx limit --pids 20 -- python3 agent/agent.py spawn 30Shown here, not run. Press Start lab to type it into your own lab’s terminal.
Try it: read a cpu.max line
# cpu.max holds two numbers: how much time the group may use in each period,
# and the period, both in microseconds.
period = 100000
for quota in [20000, 50000, 200000]:
share = quota * 100 // period
print(quota, period, "is", share, "percent of one processor")
# 200000 is more than one period: what does that mean on a machine with two processors?