Cda (counterfactual Data Augmentation)

CDA (Counterfactual Data Augmentation) is a debiasing technique that reduces social biases in language models by creating counterfactual copies of training data where demographic attributes are swapped. The idea is simple but powerful: if the model sees "The male nurse helped the patient" just as often as "The female nurse helped the patient," it cannot learn a gender association with the nursing profession.

How CDA Works

• Step 1 — Identify: Scan training text for mentions of demographic attributes — gendered pronouns (he/she), gendered nouns (king/queen, waiter/waitress), racial terms, names associated with specific demographics, etc.
• Step 2 — Swap: Create counterfactual copies of each sentence by replacing demographic terms with their counterparts:
• "She is a talented engineer" → "He is a talented engineer"
• "John received the promotion" → "Maria received the promotion"
• Step 3 — Augment: Add the counterfactual copies to the training set (either replacing originals or supplementing them).
• Step 4 — Train: Train or fine-tune the model on the augmented dataset.

Types of CDA

• Gender CDA: Swap gendered terms (most common and straightforward).
• Name-Based CDA: Swap names associated with different racial/ethnic groups.
• Multi-Attribute CDA: Swap terms across multiple bias dimensions simultaneously.

Advantages

• Intuitive: The approach is easy to understand and implement.
• Training-Time: Addresses bias at the source (training data) rather than patching it post-hoc.
• Preserves Task Performance: Usually maintains or even improves model accuracy since the augmentation provides more diverse training data.

Limitations

• Incomplete Swaps: Hard to catch all implicit gender/race signals — names, cultural references, contextual cues may be missed.
• Semantic Validity: Some swaps create implausible sentences (e.g., swapping gendered health conditions).
• Scale: Doubling the training data increases training cost.
• Binary Limitation: Simple swap-based CDA treats gender as binary and may not adequately address non-binary identities.

CDA is one of the most widely used and accessible debiasing techniques, often combined with other methods like INLP or adversarial debiasing for comprehensive bias mitigation.