What you will do
Kafka keeps messages in a topic, which is a log: producers add to the end of it and consumers read it from wherever they like. A topic is split into partitions, and every message gets an offset, its number in its own partition, counting from zero. Here you make a topic, add three messages and read them back from the beginning.
Before you start
Note
Press Start the lab on the bar at the top of this lesson and wait until it says it is running. The broker starts first and the page after it, so the first start takes about a minute. Each Run button below types its command into your lab's terminal, one after another in the same session. To watch them, or to type your own, press Open terminal on the bar.
1. Make a topic
This asks the web API behind Kafka UI for a topic called orders with three partitions. It prints 200, the web's word for "done", on a line of its own: \n after %{http_code} ends it with a new line. Then the second command lists the partitions: "partition":0, "partition":1 and "partition":2, each with "offsetMax":0. That number is the next offset, the one the next message will get, so all three are empty.
curl -s -X POST -H "Content-Type: application/json" -d '{"name":"orders","partitions":3,"replicationFactor":1,"configs":{}}' http://kafka-ui:8080/api/clusters/lab/topics -o /dev/null -w "%{http_code}\n"Shown here, not run. Create a free account to press Run and type it into your own lab’s terminal.
curl -s http://kafka-ui:8080/api/clusters/lab/topics/orders | grep -o '"partition":[0-9]*\|"offsetMax":[0-9]*'Shown here, not run. Create a free account to press Run and type it into your own lab’s terminal.
2. Produce three messages
Each command adds one message and prints 200. The first two go to partition 0 and the third to partition 1. A command has to name a partition; in the page itself, the Produce Message button on the topic does the same.
curl -s -X POST -H "Content-Type: application/json" -d '{"partition":0,"key":"alice","value":"coffee","headers":{},"keySerde":"String","valueSerde":"String"}' http://kafka-ui:8080/api/clusters/lab/topics/orders/messages -w "%{http_code}\n"Shown here, not run. Create a free account to press Run and type it into your own lab’s terminal.
curl -s -X POST -H "Content-Type: application/json" -d '{"partition":0,"key":"alice","value":"tea","headers":{},"keySerde":"String","valueSerde":"String"}' http://kafka-ui:8080/api/clusters/lab/topics/orders/messages -w "%{http_code}\n"Shown here, not run. Create a free account to press Run and type it into your own lab’s terminal.
curl -s -X POST -H "Content-Type: application/json" -d '{"partition":1,"key":"bob","value":"cake","headers":{},"keySerde":"String","valueSerde":"String"}' http://kafka-ui:8080/api/clusters/lab/topics/orders/messages -w "%{http_code}\n"Shown here, not run. Create a free account to press Run and type it into your own lab’s terminal.
3. Look at the offsets and read it back
Run the first command again: partition 0 now says "offsetMax":2, partition 1 says 1 and partition 2 still says 0. Each partition counts on its own. The second command reads the topic from the beginning, which is what a new consumer does, and prints each message's partition and offset beside its value: coffee is partition 0 offset 0, tea is partition 0 offset 1 and cake is partition 1 offset 0. They arrive newest first, so cake is listed before coffee; the offsets are the order.
curl -s http://kafka-ui:8080/api/clusters/lab/topics/orders | grep -o '"partition":[0-9]*\|"offsetMax":[0-9]*'Shown here, not run. Create a free account to press Run and type it into your own lab’s terminal.
curl -s -N "http://kafka-ui:8080/api/clusters/lab/topics/orders/messages/v2?seekType=BEGINNING&limit=10" | grep -o '"partition":[0-9]*,"offset":[0-9]*\|"value":"[a-z]*"'Shown here, not run. Create a free account to press Run and type it into your own lab’s terminal.
For teachers
Tip
This lesson is a small example of a lesson worked in a lab: the steps and the Run buttons are the lesson, and the lab is each student's own copy. The lab is two services, a broker and the page that reads it, and nothing here marks itself: the student hands in the offsets before and after and a sentence on why each partition starts at zero. The broker keeps nothing when the lab stops. Copy it into your own module and change it, or build one from scratch with the Lab menu in the lesson builder.