Art of Codeartofcode
Lesson 14 of 15

19. Search and Inverted Indexes

1
The picture to keep in your head
2
19.1 Why doesn't `LIKE '%word%'` scale?
3
19.2 How do you tokenise and normalise text?
4
19.3 What is an inverted index?
5
19.4 How does ranking work (TF-IDF, BM25 in brief)?
6
19.5 How do prefix search and tries work?
7
19.6 How do you keep the search index in sync with the database?
8
19.7 Where does Elasticsearch fit in the big picture?
9
19.8 How do you shard a search index?
10
Memory card and where the analogy breaks
11
Practice: five levels
12
Interview questions
13
Solutions
14
Sources
15
Code lab
19. Search and Inverted Indexes

Sources

3 min read+10 XP

Source notes

  • Elasticsearch default similarity BM25 with k1 = 1.2 and b = 0.75: Elasticsearch documentation, "Similarity settings"; Elastic blog, "Practical BM25 Part 2". Default index.refresh_interval of 1 second (near real-time search) and default of 1 primary shard: Elasticsearch documentation. Checked 2026-10-08.
  • Inverted indexes, TF-IDF and BM25: Manning, Raghavan and Schütze, Introduction to Information Retrieval (Cambridge University Press, 2008); Robertson and Zaragoza, "The Probabilistic Relevance Framework: BM25 and Beyond" (2009).
  • Lucene segments and merging: Apache Lucene documentation.
  • Catalogue size, row sizes, disk speed and query rates are illustrative assumptions; arithmetic computed 2026-10-08.
  • Level 3 and Level 4 code: tested 2026-10-08 (code/ch19_search.py, all checks pass).