CRDT lab: local-first sync with counters, sets, vector clocks and text in Python

This lab gives each student a JupyterLab of their own with CRDTs written in plain Python, short enough to read in full: a G-Counter and a PN-Counter, a last-writer-wins register, an observed-remove set, vector clocks, and a small text CRDT in the style of RGA with tombstones. Eleven small scripts each show one idea, from an edit lost by a naive sync to three replicas agreeing after many offline changes.

Every replica is a Python object, and offline means two objects that have not been merged yet, so nothing goes over a network and every run prints the same. Every copy has no route to the internet and none to any other student, and nothing is downloaded into it.

What students learn

Why local-first apps keep working offline, and why a sync that keeps the whole copy saved last loses edits: the lab loses a phone's title change that way. They merge G-Counters from three replicas in any order and get 10 each time, where a plain number counts a repeated message twice and reaches 19. They see a last-writer-wins register lose a write to a slow clock, an OR-set keep an item that one replica removed while another added it again, and two offline edits to one line of text merge to the same words on both replicas. They compare vector clocks as before, after, equal or concurrent, send a delta of 29 bytes instead of a state of 7890, and see which tombstones can be dropped. It suits distributed systems, databases and mobile or web development courses; check your own awarding body's unit for the exact wording, none is quoted here.

Run it in your browser

The lab starts one service, jupyter, which writes crdt.py, rga.py, a starter notebook and the short scripts into its work folder, with nothing downloaded. JupyterLab asks for no token, because the platform's own door already knows who is opening it. The terminal beside the lab has no Python of its own, so it reaches the lab's with docker exec jupyter, followed by the command to run inside.

  1. Open the Container page. Open Build on the bar and choose Run a container. Anybody may open it, and the Git lab and the SQL lab start with no account. Every other lab needs a free account: create one, sign in, then press Start.
  2. Find the lab. The Sample labs card lists every sample. Find Local-first: CRDT counters, sets and text that merge.
  3. Press Start. Start runs it as it is, in one press. Edit the file first opens the compose file so you can change it before you start.
  4. Open it. On the lab's page, the lab bar has a link called JupyterLab: press it to open JupyterLab in your browser and read crdt.py in the file editor. Then press Open terminal and type docker exec jupyter python3 steps/orset.py to see a shared list keep an item that one replica removed while another added it again.

What to ask students to hand in

A notebook of their own that uses these files to answer one question they chose, such as whether the order of merges ever changes a result, what a remove-wins set would do where the OR-set keeps an item, or how many tombstones a document keeps after a day of editing, downloaded from JupyterLab with File, Download. Then a sentence or two in their own words on what they found. Nothing is marked automatically: the sentence shows whether it was understood. For stronger work ask them to explain why a merge that takes the larger of two numbers can be repeated safely.

A basic lesson on this lab

CRDT Basics is a small lesson worked in this lab. In it a student loses an edit with a naive sync, merges counters from three replicas, keeps an item in a shared list, merges two edits to one line of text, compares vector clocks and syncs three replicas after many offline changes.

Free personal accounts have it in the Basics module, which is on Home, and every college can copy it from the Library's Basics folder. It is deliberately small: it shows a teacher what a lesson worked in a lab is, and the lab does the rest. Copy it into a module of your own to change it. The undergraduate module Local-first software and CRDTs is worked in the same lab.

The code on GitHub

The compose file has no Dockerfile: it runs one stock public image, the Jupyter Docker Stacks' own scipy-notebook, pinned to a dated tag, as its own unprivileged account with 512 MB of memory, and the CRDTs use only Python's standard library. The image is licensed BSD-3-Clause and is not distributed by us. The file writes every Python file from fixed text when it starts and the one random script uses a fixed seed, so every copy prints the same and nothing is downloaded. The limits are honest ones: the image carries no CRDT library, so Automerge and Yjs do not run here and are compared in words, and replicas are objects in one process rather than devices on a network. Nothing is kept when the lab stops, so a student downloads their notebook first.

Set it as work

Say in the task what to hand in.

  1. Make the lab. In your module add a task and choose the Container lab card, then In a GitHub repository. Give https://github.com/mitsurutech/lab-crdt-lab with the branch main. The compose file is the default, compose.yaml. Fork the repository first if you want to change it, for example to add a CRDT of your own to crdt.py or another script to steps.
  2. Publish and try it. Publish pins the commit you were shown, so a later change in the repository does not alter a lab you have already set. Start your own copy first.
  3. Hand it out. Assign it like any task, with a due date if you want one, and say in the task what to do and what to hand in.
  4. Mark what is handed in. Open a student’s copy and the work they handed in, mark it, and send it back with a reason if it needs more.

Try this lab, and what it costs

A free personal account can work through the basic lessons in this lab, with enough free credit each month to follow most lessons and no card. For a class, a school or college has its first 365 days free for up to 50 students, then pays per student a year, with every lab included and staff free.

Questions people ask

Why is there no token or password to type?
A lab's web page is reached only through the platform's own door, which already knows who you are and lets in only you and the staff who teach your module. So JupyterLab is started with no token. Do not run it like that on a machine anybody else can reach.
Why does the terminal use docker exec jupyter?
The lab's terminal carries a few clients and no Python. docker exec jupyter runs a command inside the lab's own container, which has Python. JupyterLab also has a terminal of its own, under File, New, Terminal.
Does the lab run Automerge or Yjs?
No. The lab is one stock image with nothing downloaded when it starts, and that image carries no CRDT library. The same ideas, counters, sets, vector clocks and a text CRDT with tombstones, are written in plain Python for students to read, and the lessons explain how the libraries do the same work.
Where is the network?
There is none, on purpose. Each replica is a Python object and a sync is a call to merge, so a student controls exactly which replica has heard what and every run prints the same. Going offline is simply not calling merge yet.