Hyde (hypothetical Document Embeddings)

HyDE (Hypothetical Document Embeddings) generates a hypothetical answer then searches for documents similar to it. Insight: A hypothetical answer is closer in embedding space to actual answer documents than the original question is. Process: User query → LLM generates plausible answer (may be wrong) → embed hypothetical answer → retrieve documents similar to that embedding → use retrieved docs for actual answer. Why it works: Questions and answers occupy different regions of embedding space. Hypothetical answer bridges this gap. Even incorrect hypothetical contains relevant vocabulary and structure. Implementation: Prompt LLM to answer without context, embed response, vector search, then RAG with real documents. Use cases: Particularly effective for technical domains, factual questions, when queries are very different from document style. Limitations: Extra LLM call adds latency/cost, hypothetical might mislead if very wrong. Variants: Generate multiple hypotheticals, ensemble embeddings, combine with original query embedding. Shown to improve retrieval by 10-20% on many benchmarks.