Twenty minutes of traffic in three files
Zeek is a network monitor that turns raw traffic into logs: one file for connections, one for DNS lookups, one for web requests. Analysts read these logs far more often than packets.
A packet capture of twenty minutes holds hundreds of packets. Zeek's logs hold a line for each conversation, which a person can read. You read a made-up office's traffic, in your own lab copy only.
By the end of this lesson
Note
I can read Zeek's conn, dns and http logs to see who talked to whom.
Key words
| Word | What it means | Example |
|---|---|---|
| Packet capture | a file of the packets that crossed a network | office.pcap |
| Zeek | a monitor that writes logs about network traffic | |
| Connection | one conversation between two machines, many packets long | |
| Field | one named column in a log | |
| uid | the id Zeek gives a connection, the same in every log | |
| zeek-cut | a tool that prints only the columns you name |
The cells students fill in are left blank in this preview.
The one rule
Important
Capture and read traffic only on networks you own or have written permission to watch. The office in this lab is made up and its capture is fixed. Recording somebody else's traffic without permission can be a crime.
A capture becomes three logs
Zeek reads a capture and writes conn.log, dns.log and http.log. Zeek is not in Ubuntu's archive, so the lab has zeek-lite, our small model of it. It writes Zeek's format with some of Zeek's fields.
The Runs move into the lab's folder, turn the office capture into logs, and print the line that names conn.log's columns.
Move into the lab's folder
cd ~/detectShown here, not run. Press Start lab to type it into your own lab’s terminal.
Turn the capture into logs
zeek-lite -r pcaps/office.pcapShown here, not run. Press Start lab to type it into your own lab’s terminal.
Read the names of the columns
grep "^#fields" conn.logShown here, not run. Press Start lab to type it into your own lab’s terminal.
Worked example: the main conn.log fields
| Field | What it holds | Example |
|---|---|---|
| ts | When the connection started, in seconds | 1793610030.596105 |
| id.orig_h | The machine that started it | 10.20.1.15 |
| id.resp_h | The machine that answered | 198.51.100.23 |
| id.resp_p | The port it answered on | 80 |
| service | The protocol Zeek recognised | http |
| conn_state | How it ended, SF for normally | SF |
The cells students fill in are left blank in this preview.
conn.log shows who talked to whom
zeek-cut prints only the columns you name, in your order. The first Run shows the first four connections: source, destination, port, protocol and service.
The second counts each pair of machines and sorts the busiest first. One machine, 10.20.1.15, made 20 web connections to one address. Nothing else is close.
The first four connections
zeek-cut id.orig_h id.resp_h id.resp_p proto service < conn.log | head -n 4Shown here, not run. Press Start lab to type it into your own lab’s terminal.
The busiest pairs of machines
zeek-cut id.orig_h id.resp_h service < conn.log | sort | uniq -c | sort -rn | head -n 5Shown here, not run. Press Start lab to type it into your own lab’s terminal.
A common mistake
Warning
Many people think one line of conn.log is one packet. It is one whole conversation. A web request here is eight packets: the handshake, the request, the reply and the goodbye. The fields orig_pkts and resp_pkts count them.
dns.log and http.log show the questions
dns.log has a line for each lookup: the name asked for and the answer. http.log has a line for each web request: the method, the host, the address and the user agent, the name the software gives itself.
The first Run counts every name looked up. The second counts web requests by machine and user agent.
Every name the office looked up
zeek-cut query answers < dns.log | sort | uniq -cShown here, not run. Press Start lab to type it into your own lab’s terminal.
Web requests by machine and user agent
zeek-cut id.orig_h method host uri user_agent < http.log | sort | uniq -c | sort -rn | head -n 3Shown here, not run. Press Start lab to type it into your own lab’s terminal.
Try it: read a line by name
header = "#fields\tts\tuid\tid.orig_h\tid.resp_h\tservice"
line = "1793610030.5\tCx1\t10.20.1.15\t198.51.100.23\thttp"
names = header.split("\t")[1:]
row = dict(zip(names, line.split("\t")))
print(row["id.orig_h"], "talked to", row["id.resp_h"], "over", row["service"])
# Print the uid of the connection as well.