How data replication works

Replication means keeping copies of the same data on more than one machine. If one machine breaks, another copy can take over and nothing is lost.

Almost every database and store in production is replicated. You will see how the copies are kept in step, and what a failure does to them.

In the most common design one machine is the primary. Every write goes to it. It puts each write in an ordered list, the replication log, and sends the list to the other machines.

Those machines are replicas. They replay the log in the same order, so they end up with the same data. Reads can go to any copy.

# The primary writes to its log. A replica replays the same log.
log = []
replica = []
for order in ["order 1", "order 2", "order 3"]:
    log.append(order)
# The replica copies what it has not yet seen, in order.
for entry in log:
    replica.append(entry)
print("primary log:", log)
print("replica has:", replica)
# Add "order 4" to the list in the first loop and run again.

The log is not only for replicas. A store keeps one on disk so that it can rebuild its data after a restart. Valkey calls it the append-only file: each write is added to the end and nothing is changed.

A replica is the same idea sent over the network. The store flushes its log to disk once a second by default, so a crash can lose the last second.

v() ( printf '%s\r\n' "$*" | nc -w 1 valkey 6379 )

v INFO persistence | grep aof_enabled

v CONFIG GET appendfsync

v SET ledger:7 paid

# Each copy remembers how many writes of the log it has applied.
log = ["SET a 1", "SET b 2", "SET c 3", "SET d 4", "SET e 5"]
applied = {"primary": 5, "replica1": 5, "replica2": 3}
for name in applied:
    behind = len(log) - applied[name]
    print(name, "is", behind, "writes behind")
# Change replica2 to 4 and run again.

If the primary stops, someone must pick a new one. This is failover. The best choice is the replica that has applied the most of the log, because it has lost the least.

In the table above, west is the best choice. East is 2 writes behind, and those 2 writes would be lost if east were promoted.

The primary can answer the client at once, or it can wait until a replica confirms. The first is asynchronous replication: fast, but a crash can lose writes the replica has not seen.

The second is synchronous: safe, but each write waits for a trip to the replica, and it stops if the replica is down. Most systems wait for one replica and let the rest follow.