RAG-Sequence
RAG-Sequence is a variant of Retrieval-Augmented Generation (RAG) that retrieves a set of relevant documents for a given query and generates a complete response sequence conditioned on each retrieved document independently, then marginalizes over the document distribution at the full-sequence level to produce the final output. This approach treats each retrieved document as a complete context for generation, selecting the most probable complete response across all document-conditioned generations.
Why RAG-Sequence Matters in AI/ML:
RAG-Sequence produces coherent, single-source responses that maintain narrative consistency by generating each candidate response from a single document context, then selecting the best overall response through sequence-level marginalization.
• Sequence-level marginalization — The probability of a complete output sequence y is: p(y|x) = Σ_z p(z|x) · p(y|x, z), where each candidate response is generated entirely conditioned on a single retrieved document z; the final output maximizes this marginalized probability
• Narrative coherence — Since each candidate response is generated from a single document context, RAG-Sequence produces more coherent, stylistically consistent outputs compared to RAG-Token's per-token document mixing, which can create disjointed responses
• Beam search implementation — For each retrieved document, a separate beam search generates top candidate sequences; then candidates across all documents are re-scored using the marginalized probability, selecting the globally best response
• Factual consistency — Conditioning the entire response on a single retrieved passage reduces the risk of combining contradictory facts from different sources, maintaining logical consistency within each generated response
• Retriever-generator pipeline — A DPR (Dense Passage Retrieval) bi-encoder retrieves the top-k documents, which are then individually fed to a BART generator; the final output is selected by marginalizing over retrieval scores and generation probabilities
| Aspect | RAG-Sequence | RAG-Token |
|---|---|---|
| Marginalization | Per sequence | Per token |
| Coherence | Higher (single source) | Lower (multi-source mixing) |
| Factual Consistency | Higher (one document) | Risk of contradictions |
| Multi-source Synthesis | Limited | Natural |
| Compute Cost | k × beam search | Marginalization per step |
| Best For | QA, summarization | Multi-hop synthesis |
| Implementation | Parallel beam searches | Modified attention |
RAG-Sequence provides retrieval-augmented generation with strong narrative coherence by generating complete responses conditioned on individual retrieved documents and selecting the best through sequence-level marginalization, making it ideal for tasks where factual consistency and stylistic coherence within the response are more important than multi-source information synthesis.