Context Carryover
Context carryover is the ability of a dialogue system to maintain and utilize information from previous conversation turns when processing new user messages. It is fundamental to creating natural, coherent multi-turn conversations rather than treating each message as an isolated query.
What Gets Carried Over
- Entity References: If a user says "Tell me about TSMC" then asks "What is their revenue?", the system must carry over that "their" refers to TSMC.
- Slot Values: In task-oriented dialogue, previously stated preferences (cuisine, date, budget) persist across turns without the user needing to repeat them.
- Conversation Topic: The current discussion topic provides implicit context for interpreting ambiguous queries.
- User Preferences: Learned preferences and constraints from earlier in the conversation inform later responses.
Implementation Approaches
- Full History: Pass the entire conversation history to the LLM as context. Simple but limited by context window size and can become expensive for long conversations.
- Sliding Window: Keep only the last N turns, discarding older history. Efficient but loses long-range context.
- Summarization: Periodically summarize older conversation history into a compact representation, preserving key information while reducing token usage.
- Dialogue State Tracking: Maintain a structured state object that captures all relevant information, independent of the raw conversation text.
- Memory Systems: Use vector databases or other external memory to store and retrieve relevant past context on demand.
Challenges
- Information Loss: Summarization and windowing can lose critical details from earlier in the conversation.
- Topic Shifts: Users may abruptly change topics, making older context irrelevant or even misleading.
- Ambiguity Resolution: Determining what past context is relevant to the current turn requires sophisticated understanding.
Effective context carryover is what separates a truly conversational AI from a simple question-answering system.