Every alert started as a rule
A security team gets an alert: a laptop looked up a name that a known piece of malware uses. Somebody wrote the rule that saw it, and tested it so it does not fire on every normal lookup. That work is detection engineering. Use these tools only on your own lab copy, and on a real network only with written permission.
By the end of this lesson
Note
I can run a detection rule on network traffic and read its alerts.
Key words
- Packet capture: a file of the packets that crossed a network.
- Zeek: a tool that turns traffic into logs.
- Suricata: a tool that checks traffic against rules and raises alerts.
- Sigma: a format for writing rules about logs.
- False positive: an alert on something harmless.
Starter: what you already know
Note
Answer in your head. 1. What does DNS do? 2. What is a log file? 3. Is an alert always an attack? You need the third answer in a moment.
Before you start
Note
Press Start the lab on the bar at the top of this lesson. This lab is a whole Ubuntu machine of your own, so it takes a little longer to come up than a container. Each Run button types its command into the machine's terminal, one after another in the same session. If one says the machine is still starting, wait a few seconds and press it again. To watch them, or to type your own, press Open terminal on the bar.
A capture becomes logs
Your files are in the folder detect. The first command moves into it. pcaps/office.pcap is twenty minutes of a made-up office's traffic. zeek-lite turns it into the logs Zeek writes: conn.log, dns.log and http.log. Zeek is not in Ubuntu's archive, so zeek-lite is the lab's small model of it, with the same format. The third command counts every name the office looked up.
cd detectShown here, not run. Press Start lab to type it into your own lab’s terminal.
zeek-lite -r pcaps/office.pcapShown here, not run. Press Start lab to type it into your own lab’s terminal.
zeek-cut query answers < dns.log | sort | uniq -cShown here, not run. Press Start lab to type it into your own lab’s terminal.
Check
Tip
Which names look like a tool calling home? Answer: the five under cdn-update.example, all answered 198.51.100.23. Two other names, cdn-updates.example and cdn-update.example.org, only look like them.
A rule raises alerts
rules/examples.rules holds four Suricata rules for a made-up tool called updater. det is the lab's helper: det count runs Suricata on a capture with those rules only and counts each rule's alerts. The first try fires 8 times, the tuned rule 5. The check-in rule fires 20 times, and the same rule with a limit only once.
det count pcaps/office.pcap rules/examples.rulesShown here, not run. Press Start lab to type it into your own lab’s terminal.
A common mistake
Warning
Many people think a rule that fires more is a better rule. It is not. The first try fired 8 times because 3 lookups were of names that only look like updater's. Every false positive costs somebody time, so a rule is tested before it is trusted.
Test on known-bad and known-good data
det test runs the rules on two captures whose answers are known. known-bad.pcap holds five bad events and known-good.pcap only normal traffic. A good rule fires on all of the first and none of the second. Only the first try fires on known-good, 3 times.
det test rules/examples.rulesShown here, not run. Press Start lab to type it into your own lab’s terminal.
Check
Tip
Which rule needs tuning, and why? Answer: rule 1000001, because it alerted 3 times on known-good traffic. Those are false positives.
A Sigma rule searches logs
sigma/whoami.yml is a Sigma rule for Windows events: it matches any run of whoami. det sigma converts it into SQL with the real Sigma tools and runs it on the lab's events. It finds two: a user on PC-15 straight after a suspicious PowerShell, and an admin on PC-13 doing their job.
det sigma sigma/whoami.ymlShown here, not run. Press Start lab to type it into your own lab’s terminal.
Exit check
Tip
Answer without looking back. 1. What does Zeek turn traffic into? 2. What does Suricata do with a rule? 3. What is a false positive? 4. Why test on known-good data? Answers: logs; checks traffic against it and raises alerts; an alert on something harmless; to see the false positives. Now open README.md in your home folder with cat ~/README.md: you write rules for a made-up tool called quietbird.
For teachers
Tip
This lesson is a small example of a lesson worked in a lab: the steps and the Run buttons are the lesson, and the lab is each student's own copy. This lesson is worked in a VM lab: each student has an Ubuntu machine of their own with Suricata, the Sigma tools and the lab's model of Zeek, and a Run types into its terminal and shows what it printed. Key learning points: 1. Zeek-style logs summarise traffic. 2. A Suricata rule raises alerts on matching traffic. 3. A rule is tested on known-bad and known-good data. 4. Sigma rules search logs. Common mistake: thinking more alerts means a better rule. Equipment: the lab and a browser, and a free or college account, since the setup installs from PyPI. Tip: ask the class to guess which names are look-alikes before the count. The lab itself has a check that marks two files with traffic and logs it makes afresh. Copy it into your own module and change it, or build one from scratch with the Lab menu in the lesson builder.