Name Substitution

Name substitution is the fairness evaluation and augmentation technique that replaces personal names to probe demographic sensitivity in model behavior - it helps detect bias tied to ethnicity, gender, or cultural identity signals.

What Is Name substitution?

• Definition: Paired-text transformation where only personal names are changed while context remains constant.
• Evaluation Purpose: Measure whether outputs differ due to demographic proxy cues from names.
• Augmentation Use: Build more demographically balanced training examples.
• Method Constraint: Substitutions must preserve semantics and pragmatic plausibility.

Why Name substitution Matters

• Bias Auditing: Exposes unequal model treatment associated with identity-coded names.
• Fairness Improvement: Supports targeted data interventions where name-linked bias is observed.
• Causal Clarity: Paired tests isolate demographic signal effects from content differences.
• Risk Reduction: Helps prevent discriminatory behavior in user-facing applications.
• Benchmark Alignment: Useful for evaluating progress on fairness metrics over model versions.

How It Is Used in Practice

• Name Sets: Use curated balanced name lists with documented demographic coverage.
• Paired Scoring: Compare probabilities, classifications, and generated sentiment across substitutions.
• Mitigation Feedback: Feed detected disparities into retraining and policy refinement.

Name substitution is a practical fairness-testing instrument in LLM evaluation - controlled identity-proxy swaps provide actionable evidence for detecting and correcting demographic bias patterns.