Art of Codeartofcode
Lesson 14 of 15

15. Sharding

1
The picture to keep in your head
2
15.1 Why shard at all?
3
15.2 How do you choose the shard key?
4
15.3 What are the ways to map keys to shards?
5
15.4 What is a hot shard?
6
15.5 How do you add shards without moving everything?
7
15.6 What happens to queries and joins that cross shards?
8
15.7 How do you keep IDs and unique rules across shards?
9
15.8 How do sharding and replication work together?
10
Memory card and where the analogy breaks
11
Practice: five levels
12
Interview questions
13
Solutions
14
Sources
15
Code lab
15. Sharding

Sources

3 min read+10 XP

Source notes

  • Logical shards mapped onto fewer physical servers, and shard IDs inside object IDs: Instagram Engineering, "Sharding & IDs at Instagram", 2012.
  • Range-partitioned tablets that split as they grow: Fay Chang et al., "Bigtable: A Distributed Storage System for Structured Data", OSDI 2006. Hash-partitioned rings: DeCandia et al., "Dynamo", SOSP 2007, and Apache Cassandra documentation.
  • Redis Cluster's 16,384 hash slots: Redis Cluster specification.
  • Vitess (sharding and resharding for MySQL) and Citus (distributed PostgreSQL): project documentation.
  • All bookstore numbers are illustrative assumptions; arithmetic computed 2026-10-08.
  • Level 3 and Level 4 code: tested 2026-10-08 (code/ch15_sharding.py, all checks pass).