An alert is somebody's rule
Detection engineering is the work of writing, testing and looking after the rules that turn logs and network traffic into alerts. Every alert a security team reads exists because somebody wrote a rule for it.
A good rule catches an attack early. A careless one buries the team in noise. Before anything else, one rule of ours: work only on your own lab copy, and on a real network only with written permission.
By the end of this lesson
Note
I can follow a detection through its lifecycle, from hypothesis to retirement.
Key words
| Word | What it means | Example |
|---|---|---|
| Detection | a rule that raises an alert when it sees certain behaviour | |
| Hypothesis | a guess about what an attack leaves behind, which can be tested | |
| False positive | an alert on something harmless | |
| Tuning | changing a rule so it makes fewer mistakes | |
| Detection as code | keeping rules as files with review, tests and history | |
| SID | the number that names one Suricata rule | 1000002 |
The cells students fill in are left blank in this preview.
The one rule of this module
Important
Every capture and log in this module was made up for the lab. Run these tools only on your own lab copy. On a real network, watch traffic only with written permission from its owner. The point is defence: you learn to see an attack so it can be stopped.
A detection starts as a hypothesis
Nobody writes a good rule by staring at logs. You start from a hypothesis: if this attack happened here, what would it leave behind?
Suppose a report says a tool called updater looks up names under cdn-update.example and then calls home over the web. The hypothesis is: an infected machine sends DNS lookups for those names.
Worked example: from a report to a rule
| Step | For updater |
|---|---|
| What the report says | It looks up names under cdn-update.example |
| Hypothesis | An infected machine sends those DNS lookups |
| Where to look | DNS traffic on the office network |
| Rule | Alert on a DNS query for that domain |
| Test | Run the rule on traffic where the answer is known |
The cells students fill in are left blank in this preview.
The lab keeps its rules as files
The lab has a folder of example rules for updater, written for Suricata, a tool that reads network traffic and raises alerts.
The first Run moves into the lab's folder. The second shows the lab's helper, det. The third counts the rules in the file, and the fourth prints the first rule, SID 1000001.
Move into the lab's folder
cd ~/detectShown here, not run. Press Start lab to type it into your own lab’s terminal.
See what the det helper does
detShown here, not run. Press Start lab to type it into your own lab’s terminal.
Count the example rules
grep -c "^alert" rules/examples.rulesShown here, not run. Press Start lab to type it into your own lab’s terminal.
Read the first rule
grep "sid:1000001" rules/examples.rulesShown here, not run. Press Start lab to type it into your own lab’s terminal.
Testing shows what a rule really does
A rule then goes round a loop. You test it on traffic where you know the answer. If it misses attacks or fires on harmless things, you tune it and test again.
The Run counts each example rule's alerts on twenty minutes of office traffic. The first try fires 8 times and the tuned rule 5. The difference is 3 lookups of names that only look like updater's.
Count each rule's alerts
det count pcaps/office.pcap rules/examples.rulesShown here, not run. Press Start lab to type it into your own lab’s terminal.
A common mistake
Warning
Many people think a rule that fires more is a better rule. It is not. Every false positive costs an analyst time. A rule is good when it fires on the attack and stays quiet otherwise. That is why every rule is tested before it is trusted.
Detection as code keeps rules honest
Teams keep rules as plain files in version control. A change is reviewed like code, and the tests run every time a rule changes.
A rule is also retired when its threat has gone or it has not fired usefully for months. A rule nobody maintains still costs time to run and to read.
Try it: the next stage
stages = ["hypothesis", "rule", "test", "tune", "retire"]
rule = {"name": "updater lookup", "stage": "rule"}
at = stages.index(rule["stage"])
print(rule["name"], "is at", rule["stage"])
print("next:", stages[at + 1])
# Change the stage to "test". What comes next?