17.9 What are distributed locks, and why are they tricky?

On the train: Two clerks in different stations both want to sell seat 14. They phone a central desk that hands out one "talking stick" per seat. Simple, until a clerk with the stick falls asleep for ten minutes, the desk assumes they're gone and gives the stick to someone else, and the sleeping clerk wakes up and sells the seat anyway.

In the system: A distributed lock lets processes on different machines agree that only one of them works on something at a time (only one worker sends tonight's newsletter; only one process refreshes a cache key, Chapter 13). Common ways:

• Redis: SET lock:seat14 <token> NX PX 30000: set only if absent, expire after 30 seconds so a crashed holder doesn't block forever. Release only if the token still matches.
• Coordination services like ZooKeeper or etcd, with leases tied to a session.
• The database itself: a row lock or an advisory lock, often the simplest when you already have one database.

Why they're tricky: a lock with an expiry can expire while the holder is still working, for example during a long garbage-collection pause or a network hiccup. Now two processes believe they hold the lock. The classic protection is a fencing token: every grant comes with an increasing number, and the resource (the database, the storage) rejects writes carrying an older number than one it has already seen. Martin Kleppmann's 2016 essay "How to do distributed locking" is the well-known discussion of this problem.

Rule of thumb: if correctness truly depends on the lock (money, bookings), prefer the database's own guarantees (unique constraints, transactions, row locks) over a separate lock service, and use distributed locks for efficiency (avoiding duplicate work) rather than for correctness.

Memory code: "A lock that can expire can be held twice, so number every holder."

---