16. Consistency and CAP
The picture to keep in your head
Every question below is answered twice: first at the bank counters, so you can picture it, then in the system, so you know exactly how it works. Each answer ends with a memory code to lock it in.
The picture to keep in your head
A bank has two counters in different towns, both serving the same account. Normally the clerks phone each other before every withdrawal: "Asha wants ₹400; is her balance still ₹500?" Then one day the phone line is cut. Asha walks up to counter 2 and asks for ₹400. The clerk has a choice. Refuse ("come back when the line is fixed"), which is safe but leaves Asha without cash. Or pay out from the last balance they knew, which keeps Asha happy but risks paying money that was already withdrawn at counter 1.
And even on a normal day, phoning first makes every withdrawal slower.
That's the whole of this chapter: consistency (do all copies agree?), what happens when copies can't talk (CAP), and what consistency costs on a normal day (PACELC).
Our running example: the bookstore's gift-card wallet, now replicated in Mumbai and Frankfurt (Chapter 14). A customer can spend their gift-card balance from either region.
| At the bank | In the system |
|---|---|
| Two counters, one account | Two replicas of the same data |
| Phoning before every withdrawal | Coordinating replicas before answering |
| The line being cut | A network partition |
| "Come back later" | Choosing consistency: return an error rather than a wrong answer |
| Paying from the last known balance | Choosing availability: answer, maybe stale |
| Every withdrawal taking longer because of the call | The latency cost of consistency (PACELC) |
Figure: when the line is cut, each counter must choose: refuse the customer, or risk a wrong answer.