Limitations
LLM Limitations and Boundaries
Fundamental Limitations
Knowledge Cutoff
LLMs have training data cutoff dates:
| Model | Knowledge Cutoff |
|-------|------------------|
| GPT-4o | Varies by version |
| Claude 3 | Early 2024 |
| Llama 3 | December 2023 |
Implication: Cannot answer about recent events without retrieval.
Context Window Constraints
- Maximum tokens per request (e.g., 128K, 200K)
- "Lost in the middle" problem for very long contexts
- Cost scales with context length
Hallucinations
LLMs may generate:
- Plausible-sounding but false information
- Non-existent citations or references
- Confident answers about things they do not know
What LLMs Cannot Do Well
Reliable Computation
| Task | Problem | Workaround |
|------|---------|------------|
| Complex math | May make arithmetic errors | Use code execution |
| Counting | Inconsistent for large sets | Use programmatic counting |
| Logical proofs | May skip steps or err | Verify with formal tools |
Real-time Information
- No access to current events
- Cannot check live stock prices, weather
- Solution: Tool use, RAG with current data
Precision Tasks
| Task | Issue | Better Approach |
|------|-------|-----------------|
| Exact text matching | May paraphrase | Use regex/code |
| Character counting | Tokenization obscures | Use len() |
| Consistent formatting | May drift | Use structured output |
Guaranteed Safety
- Jailbreaks and prompt injection possible
- Cannot guarantee 100% filter compliance
- Requires defense in depth approach
Things to Be Careful About
High-Stakes Decisions
❌ LLMs should not be sole deciders for:
- Medical diagnoses
- Legal advice
- Financial decisions
- Safety-critical systems
✅ Use as assistants with human oversight
Private Information
- LLMs may memorize training data
- API calls may be logged by providers
- Consider privacy implications
Consistency
- Same prompt may give different outputs
- Temperature=0 helps but not guaranteed
- For critical consistency, verify programmatically
Mitigation Strategies
For Hallucinations
1. Use RAG with verified sources
2. Request citations and verify them
3. Cross-check with multiple queries
4. Add fact-checking step
For Math/Logic
1. Use code execution tools
2. Chain-of-thought prompting
3. Self-consistency (multiple samples)
4. Formal verification where possible
For Safety
1. Layer multiple guardrails
2. Content filtering on input/output
3. Human review for sensitive content
4. Rate limiting and monitoring