Easier to read with cream paper, a tint, more contrast or larger text? Choose your colours and text, once, for every page.

Free module

Big data engineering

Learn big data engineering hands on: Hadoop and MapReduce, Spark, Kafka streams and key-value stores, in real labs.

4 lessons215 min lessonSoftware engineering

The lessons

Each lesson is free to read in full. Create a free account to run its commands in a live lab and to sit its quizzes and tasks.

  1. 1 · Container lab

    Splitting a big file across machines (HDFS)

    I can explain how a file is split and processed in parallel.

    Read the lesson

  2. 2 · Container lab

    Running a data job on many machines (Spark)

    I can express a data job as a Spark pipeline.

    Read the lesson

  3. 3 · Container lab

    Passing events between programs (Kafka)

    I can explain a log of events and how consumers read it.

    Read the lesson

  4. 4 · Container lab

    Fast lookups by key (key-value stores)

    I can choose a key-value store for the right job.

    Read the lesson

Worked in a real lab

Every lesson of this module is worked in a lab started for you alone. The labs it uses:

Teaching a class? Set this module for your students with a teacher guide, marked assignments and a lab for every student.