Hnsw
HNSW (Hierarchical Navigable Small World) is an approximate nearest neighbor algorithm optimized for high-dimensional vector search — providing sub-millisecond query times on millions of vectors through a multi-layer graph structure, making it the foundation of modern vector databases.
What Is HNSW?
- Type: Approximate nearest neighbor (ANN) search algorithm.
- Structure: Multi-layer graph with skip-list-like hierarchy.
- Speed: Sub-millisecond queries on millions of vectors.
- Accuracy: 95-99% recall with proper tuning.
- Usage: Core algorithm in Qdrant, Milvus, Pinecone, FAISS.
Why HNSW Matters
- Speed: 100-1000× faster than brute-force search.
- Scalability: Handles billions of vectors efficiently.
- Accuracy: High recall rates for production use.
- Memory-Efficient: Optimized graph structure.
- Industry Standard: Used by all major vector databases.
How It Works
1. Build Phase: Insert vectors into multi-layer graph.
2. Layers: Top layers have few nodes (long jumps), bottom layers dense (fine search).
3. Search: Start at top layer, greedily descend to find nearest neighbors.
4. Result: Fast approximate nearest neighbors with tunable accuracy.
Key Parameters
- M: Number of connections per node (higher = more accurate, slower).
- ef_construction: Build-time search depth.
- ef_search: Query-time search depth.
HNSW is the backbone of semantic search — enabling real-time similarity search at scale.