The price list that vanished
A small online shop has a page that shows its price list. One morning the page says "File not found".
The owner is sure the file is still there. Is she right? Guessing will not tell you. The machine keeps a record of what went wrong, if you know where to look.
By the end of this lesson
Note
I can find a fault from a log and fix it.
Key words
| Word | What it means | Example |
|---|---|---|
| Fault | something wrong that stops a system working | |
| Symptom | what a user sees when there is a fault | |
| Log | a file where a program writes what it did, with the time | |
| Journal | the log systemd keeps for every service | journalctl |
| Status code | the number a web server answers with | 404 |
The cells students fill in are left blank in this preview.
Before you start
Note
Press Start the lab on the bar at the top and wait for the machine. This lab has a log that grows every two seconds and a small web server on port 8000. Press the Runs in order.
Read the log before you guess
A log line starts with the date and time. Then comes a level, such as INFO for normal work or ERROR for a problem. Then comes a message.
The newest lines are at the end, so tail shows them. grep -c ERROR counts the error lines. A pipe into tail -n 2 shows only the two newest errors.
The three newest lines
tail -n 3 /var/log/test.logShown here, not run. Create a free account to press Run and type it into your own lab’s terminal.
How many errors so far?
grep -c ERROR /var/log/test.logShown here, not run. Create a free account to press Run and type it into your own lab’s terminal.
The two newest errors
grep ERROR /var/log/test.log | tail -n 2Shown here, not run. Create a free account to press Run and type it into your own lab’s terminal.
Worked example: four steps to a fix
| Step | Ask | At the shop |
|---|---|---|
| 1 Symptom | What does the user see? | The page says File not found |
| 2 Evidence | What does the log say? | The server says code 404 |
| 3 Cause | Why did that happen? | Nobody may read the file |
| 4 Fix and check | Change one thing, then test again | chmod 644, then load the page |
The cells students fill in are left blank in this preview.
Make the fault, then find it
The lab's web server shares a file, report.csv. The first Run breaks it on purpose: chmod 000 takes every right away.
Then work like somebody who was not told. curl -sI asks the server for the file and shows only its answer. journalctl -u lab-web reads the server's own log.
Break it on purpose
sudo chmod 000 /srv/lab-web/report.csvShown here, not run. Create a free account to press Run and type it into your own lab’s terminal.
What does the server answer?
curl -sI http://localhost:8000/report.csv | head -n 1Shown here, not run. Create a free account to press Run and type it into your own lab’s terminal.
Read the server's own log
sudo journalctl -u lab-web -n 2 --no-pager -o catShown here, not run. Create a free account to press Run and type it into your own lab’s terminal.
Is the file really gone?
ls -l /srv/lab-webShown here, not run. Create a free account to press Run and type it into your own lab’s terminal.
A common mistake
Warning
Many people believe an error message word for word. "File not found" sounded like a missing file, but the file was there. A message says what the program thinks went wrong. Check the real thing, here with ls -l, before you fix anything.
Fix it: let everybody read it
sudo chmod 644 /srv/lab-web/report.csvShown here, not run. Create a free account to press Run and type it into your own lab’s terminal.
Check the fix
curl -sI http://localhost:8000/report.csv | head -n 1Shown here, not run. Create a free account to press Run and type it into your own lab’s terminal.
Going further
Try a second fault. Run sudo systemctl stop lab-web yourself, then forget you did.
Now find it with the four steps. curl -sS http://localhost:8000/hello.txt shows the symptom. systemctl is-active lab-web and the journal give the evidence. Write down the cause and the fix, then fix it and check.