Art of Codeartofcode
Lesson 6 of 17

14. Replication

1
The picture to keep in your head
2
14.1 Why keep copies of the same data?
3
14.2 How does leader-follower replication work?
4
14.3 Should the leader wait for followers before confirming a write?
5
14.4 What is replication lag, and why does it happen?
6
14.5 How do you stop lag from confusing users?
7
14.6 How do you use read replicas in practice?
8
14.7 What happens when the leader fails?
9
14.8 What is split brain, and how is a leader safely chosen?
10
14.9 When would you have more than one leader?
11
14.10 How does leaderless replication work, and what are quorums?
12
Memory card and where the analogy breaks
13
Practice: five levels
14
Interview questions
15
Solutions
16
Sources
17
Code lab
14. Replication

14.5 How do you stop lag from confusing users?

3 min read+10 XP

At the bank: A customer changes their address at head office, then walks to a branch to check. The branch's copy hasn't arrived, so it shows the old address. The customer panics. Smart fix: for the next hour, check that customer's details at head office, not at the branch.

In the system: Lag causes a few specific, confusing bugs. Each has a fix.

Asha saves a new address, then reloads from a lagging replica and sees the old one.

Figure: the replica is fine; it's just 300 ms behind. The user thinks her change was lost.

  1. Read-your-writes (read-after-write): a user must always see their own latest changes. Fixes:
    • Read from the leader for data the user may have just changed (their own profile, cart, orders), or for a few seconds after any write by that user.
    • Remember the log position of the user's last write; serve their reads from a follower only if it has applied at least that position (the Level 3 code does this).
  2. Monotonic reads: a user mustn't see newer data and then older data, which can happen if two reads hit two followers with different lag. Fix: send each user's reads to the same follower (for example, chosen by hashing the user ID).
  3. Consistent prefix: someone must never see an answer before its question (a reply to a comment that doesn't exist yet). Fix: write related data so it's replicated together (same transaction or same partition).

These are guarantees for one user's view; Chapter 16 covers consistency models more fully.

Memory code: "Show people their own changes, and never let time run backwards."