Hierarchical Navigable Small World indexing built by hand: multilayer graphs, greedy descent, heuristic pruning, custom distance kernels, persistence and recall-vs-latency benchmarks.
I had used vector databases without understanding what the index actually did or why recall degraded when parameters changed. Building HNSW from the paper — no library — was the way to find out where the accuracy and the latency really come from.
Me, as the reader of my own retrieval stack
Know what ef, M and layer assignment actually trade against each other before tuning a production index.
Any RAG system I build
A defensible answer to 'why this recall at this latency' instead of vendor defaults.
Each node gets a maximum layer from an exponentially decaying distribution: a few nodes sit high and act as long-range hubs, most exist only at the base. Search starts at the sparse top and descends, so the upper layers do the coarse travel and the base does the fine work.
At every layer above the base, search is plain greedy — move to a closer neighbour until none is closer, then drop down. At the base it becomes a beam search with a candidate heap of width ef. That single parameter is the recall/latency dial, and seeing it move the curve made it concrete.
Taking the M nearest neighbours produces clustered edges and a graph that traps greedy search in local minima. The heuristic keeps a candidate only if it is closer to the new node than to any already-selected neighbour, which preserves diverse directions and keeps long-range navigability.
Cosine and Euclidean written directly, with vectors pre-normalised where the metric allows it so the inner loop is a dot product. Distance calls are the hot path — counting them per query turned out to be more informative than wall-clock timing alone.
The graph is serialised — vectors, adjacency per layer, entry point, parameters — and loaded back without a rebuild, so a benchmark run does not start with an index build.
Recall@k measured against brute-force ground truth, with latency percentiles across M, ef_construction and ef_search. The output is a curve, not a single number: for this dataset, this is what each parameter buys and what it costs.