Bold

BOLD is the Bias in Open-Ended Language Generation benchmark that evaluates social bias patterns in free-form model outputs across demographic domains - it focuses on bias in generation rather than only classification tasks.

What Is BOLD?

• Definition: Prompt-based benchmark for measuring sentiment and regard patterns in open-ended generated text.
• Domain Coverage: Includes demographic categories such as profession, gender, race, religion, and ideology contexts.
• Evaluation Style: Analyze generated continuations for positivity, negativity, and representational bias signals.
• Model Relevance: Targets generative systems where output framing can encode subtle stereotypes.

Why BOLD Matters

• Generation-Focused Fairness: Captures bias behavior in realistic free-text outputs.
• Risk Visibility: Reveals tone disparities that may not appear in closed-form benchmarks.
• Mitigation Feedback: Useful for assessing alignment and debiasing effects on open-ended generation.
• User Impact: Generated sentiment bias directly affects perceived fairness and trust.
• Evaluation Complement: Adds coverage beyond pairwise and coreference-only fairness tests.

How It Is Used in Practice

• Prompt Sampling: Generate outputs for benchmark prompts under controlled decoding settings.
• Metric Analysis: Compute regard and sentiment distributions by demographic category.
• Longitudinal Tracking: Monitor BOLD trends across model versions and safety updates.

BOLD is a key benchmark for bias assessment in open-ended language generation - domain-level sentiment and regard analysis helps identify representational harms in real conversational and content-generation use cases.