Conversation

Multi-Turn Conversations are the stateless simulation of persistent dialogue achieved by including complete conversation history in every API call — requiring developers to explicitly manage conversation state, context window budgets, and history truncation strategies because language models have no built-in memory between API calls and must reconstruct context from the provided message array on every request.

What Is a Multi-Turn Conversation?

• Definition: A sequence of alternating user and assistant messages where each turn builds on prior context — the AI remembers what was said, refers to previous topics, and maintains coherent dialogue across multiple exchanges.
• The Fundamental Illusion: LLMs are stateless functions — f(messages) → response. They have no memory, no session state, no persistent knowledge of previous calls. Every "memory" in a conversation is achieved by re-sending the entire history.
• Developer Responsibility: Unlike traditional databases that persist state automatically, multi-turn AI conversations require the application layer to explicitly manage, store, and re-transmit conversation history on every turn.
• Context Window Budget: The conversation history consumes the model's context window — a 128K token model can hold roughly 90,000-100,000 tokens of conversation before history must be pruned.

Why Multi-Turn Conversation Management Matters

• Coherence: Without proper history management, models cannot refer to earlier parts of the conversation, answer follow-up questions correctly, or maintain consistent persona and decisions.
• Cost: Each turn re-sends the entire history — a 10-turn conversation at turn 10 sends 9x the tokens of turn 1. Input token costs compound multiplicatively.
• Latency: Longer context windows take longer to process — first-token latency increases with conversation length.
• Context Window Limits: 4K, 8K, 32K, 128K token limits constrain how much history can be maintained — requiring management strategies for long conversations.
• Relevance Decay: Early conversation turns may become irrelevant as conversation evolves — naive FIFO truncation drops important early context (user's initial problem statement).

Multi-Turn Implementation Pattern

conversation_history = []

def chat(user_message: str, system_prompt: str) -> str:
    # Add user message to history
    conversation_history.append({"role": "user", "content": user_message})

    # Build complete message array (system + full history)
    messages = [{"role": "system", "content": system_prompt}] + conversation_history

    # Call API with full history
    response = client.chat.completions.create(
        model="gpt-4o",
        messages=messages
    )

    # Extract and store assistant response
    assistant_message = response.choices[0].message.content
    conversation_history.append({"role": "assistant", "content": assistant_message})

    return assistant_message

Context Management Strategies

Naive Truncation (FIFO):
- Drop oldest messages when context window fills.
- Simple to implement, but loses critical early context (initial problem statement, user preferences).
- Best for: simple Q&A sessions without complex dependencies.

Smart Truncation (Preserve Anchors):
- Always keep: system prompt + first user message + last N turns.
- Drop: middle turns when context fills.
- Better for: conversations with important setup context in early turns.

Summarization:
- When history exceeds threshold, summarize old turns: "Summarize this conversation in 200 words preserving key decisions and context."
- Insert summary as system context; discard summarized turns.
- Best for: long conversations where summarized context suffices.

Vector Memory:
- Store all turns as embeddings in a vector database.
- On each turn, retrieve the K most semantically relevant prior turns.
- Inject retrieved context into the current prompt.
- Best for: very long sessions (days/weeks) where exact history retrieval is too large for context.

Context Window Usage by Model

Token Cost Implications

In a 20-turn conversation with 200 tokens per turn:
- Turn 1: 200 input tokens
- Turn 10: 2,000 input tokens (full history)
- Turn 20: 4,000 input tokens (full history)
- Total input tokens: ~42,000 (sum of 200+400+...+4000)

At GPT-4o pricing ($5/1M input tokens): ~$0.21 for a 20-turn conversation — manageable, but in production systems with thousands of concurrent conversations, these costs compound.

Multi-turn conversations are the foundational interaction paradigm for AI assistants — but beneath the seamless dialogue experience lies a stateless function repeatedly consuming growing context windows, and managing this architecture efficiently — through smart truncation, summarization, and vector memory — is what separates prototype chatbots from production-grade AI applications.