Easier to read with cream paper, a tint, more contrast or larger text? Choose your colours and text, once, for every page.

Free lesson

Reading Zeek logs from a packet capture

After this lesson: I can read Zeek's conn, dns and http logs to see who talked to whom.

60 min lesson7 commandsDetection engineering lab: Suricata rules, Sigma rules and Zeek-style logs on one private Ubuntu VM

Twenty minutes of traffic in three files

Zeek is a network monitor that turns raw traffic into logs: one file for connections, one for DNS lookups, one for web requests. Analysts read these logs far more often than packets.

A packet capture of twenty minutes holds hundreds of packets. Zeek's logs hold a line for each conversation, which a person can read. You read a made-up office's traffic, in your own lab copy only.

By the end of this lesson

Note

I can read Zeek's conn, dns and http logs to see who talked to whom.

Key words

WordWhat it meansExample
Packet capturea file of the packets that crossed a networkoffice.pcap
Zeeka monitor that writes logs about network traffic
Connectionone conversation between two machines, many packets long
Fieldone named column in a log
uidthe id Zeek gives a connection, the same in every log
zeek-cuta tool that prints only the columns you name

The cells students fill in are left blank in this preview.

The one rule

Important

Capture and read traffic only on networks you own or have written permission to watch. The office in this lab is made up and its capture is fixed. Recording somebody else's traffic without permission can be a crime.

A capture becomes three logs

Zeek reads a capture and writes conn.log, dns.log and http.log. Zeek is not in Ubuntu's archive, so the lab has zeek-lite, our small model of it. It writes Zeek's format with some of Zeek's fields.

The Runs move into the lab's folder, turn the office capture into logs, and print the line that names conn.log's columns.

Move into the lab's folder

Command
cd ~/detect

Shown here, not run. Press Start lab to type it into your own lab’s terminal.

Turn the capture into logs

Command
zeek-lite -r pcaps/office.pcap

Shown here, not run. Press Start lab to type it into your own lab’s terminal.

Read the names of the columns

Command
grep "^#fields" conn.log

Shown here, not run. Press Start lab to type it into your own lab’s terminal.

Worked example: the main conn.log fields

FieldWhat it holdsExample
tsWhen the connection started, in seconds1793610030.596105
id.orig_hThe machine that started it10.20.1.15
id.resp_hThe machine that answered198.51.100.23
id.resp_pThe port it answered on80
serviceThe protocol Zeek recognisedhttp
conn_stateHow it ended, SF for normallySF

The cells students fill in are left blank in this preview.

conn.log shows who talked to whom

zeek-cut prints only the columns you name, in your order. The first Run shows the first four connections: source, destination, port, protocol and service.

The second counts each pair of machines and sorts the busiest first. One machine, 10.20.1.15, made 20 web connections to one address. Nothing else is close.

The first four connections

Command
zeek-cut id.orig_h id.resp_h id.resp_p proto service < conn.log | head -n 4

Shown here, not run. Press Start lab to type it into your own lab’s terminal.

The busiest pairs of machines

Command
zeek-cut id.orig_h id.resp_h service < conn.log | sort | uniq -c | sort -rn | head -n 5

Shown here, not run. Press Start lab to type it into your own lab’s terminal.

A common mistake

Warning

Many people think one line of conn.log is one packet. It is one whole conversation. A web request here is eight packets: the handshake, the request, the reply and the goodbye. The fields orig_pkts and resp_pkts count them.

dns.log and http.log show the questions

dns.log has a line for each lookup: the name asked for and the answer. http.log has a line for each web request: the method, the host, the address and the user agent, the name the software gives itself.

The first Run counts every name looked up. The second counts web requests by machine and user agent.

Every name the office looked up

Command
zeek-cut query answers < dns.log | sort | uniq -c

Shown here, not run. Press Start lab to type it into your own lab’s terminal.

Web requests by machine and user agent

Command
zeek-cut id.orig_h method host uri user_agent < http.log | sort | uniq -c | sort -rn | head -n 3

Shown here, not run. Press Start lab to type it into your own lab’s terminal.

Try it: read a line by name

Code: python
header = "#fields\tts\tuid\tid.orig_h\tid.resp_h\tservice"
line = "1793610030.5\tCx1\t10.20.1.15\t198.51.100.23\thttp"

names = header.split("\t")[1:]
row = dict(zip(names, line.split("\t")))
print(row["id.orig_h"], "talked to", row["id.resp_h"], "over", row["service"])
# Print the uid of the connection as well.

About this lab and its source: Detection engineering lab: Suricata rules, Sigma rules and Zeek-style logs on one private Ubuntu VM

Teaching a class? Set this module for your students with a teacher guide, marked assignments and a lab for every student.