Easier to read with cream paper, a tint, more contrast or larger text? Choose your colours and text, once, for every page.

Free resources

Data engineering

Distributed storage, batch processing, streams and an in-memory store, on your own private copy.

← All topics

Hands-on labs

Labs

Every lab is a real environment with a terminal beside it, nothing to install. Each has a short basic lesson that shows how a lesson works with a lab. A free account runs every lab and is given the basic lessons on Home; with no account you can run the Git lab and the SQL lab.

Container lab

Hadoop lab: HDFS and MapReduce, with a file to put in and a word count to read

Each student gets a private Hadoop cluster: put a file into HDFS, list it, read it back and read a MapReduce word count. Source on GitHub.

Free lessons

  • Hadoop Basics

How it works · Source on GitHub · Create a free account to run it

Container lab

Apache Spark lab: learn DataFrames and Spark SQL on a CSV file, with the Spark UI

A private Apache Spark for each student: load a CSV into a DataFrame, filter, group, query in SQL and see the jobs in the Spark UI. Code on GitHub.

Free lessons

  • Spark Basics

How it works · Source on GitHub · Create a free account to run it

Container lab

Kafka lab: topics, producers, consumers, partitions and offsets in the browser

Each student gets a private Apache Kafka broker and Kafka UI: make a topic, produce messages, read them from the beginning and see offsets. Source on GitHub.

Free lessons

  • Kafka Basics

How it works · Source on GitHub · Create a free account to run it

Container lab

Valkey lab: learn a Redis-compatible key-value store with keys, expiry, lists and counters

A private Valkey store, Redis-compatible, for each student: keys, expiry, lists, hashes and counters in a browser terminal. Nothing to install; code on GitHub.

Free lessons

  • Valkey Basics

How it works · Source on GitHub · Create a free account to run it

Basic lessons are given to every free account, on Home, and to every college. Curriculum lessons, such as T Level, are for colleges. Labs marked as an Ubuntu VM lab are set by a teacher in a module, and their lessons start the VM for you.

Run these yourself

A free personal account runs every lab here and its lesson. Teaching a class? Start a college and set them for your students.

Make a free accountStart a college

Already have an account? Sign in