Stopping Criteria

Stopping criteria is the formal set of rules used by the decoder to decide when generation should end for a request - they define termination behavior across all decoding modes.

What Is Stopping criteria?

• Definition: Configured conditions that signal completion of generation.
• Criterion Types: Length limits, EOS detection, stop strings, grammar completion, and timeout guards.
• Scope: Applied consistently across greedy, beam, and sampling decoders.
• Implementation Point: Evaluated each decode step before requesting next token.

Why Stopping criteria Matters

• Consistency: Standard criteria ensure predictable response boundaries.
• Resource Control: Prevent runaway generation that exhausts token budgets.
• Format Reliability: Support strict outputs like JSON and template-constrained text.
• Latency Governance: Termination rules contribute directly to SLA compliance.
• Safety Assurance: Stops generation when policy or execution limits are reached.

How It Is Used in Practice

• Policy Specification: Document stop priorities and precedence across multiple criteria.
• Edge-Case Validation: Test nested stops, partial matches, and multilingual tokenization effects.
• Runtime Audits: Log triggered criterion type for each completion to detect anomalies.

Stopping criteria is the termination contract of any production decoding pipeline - robust criteria prevent truncation bugs and uncontrolled output growth.