Multilingual Alignment

Multilingual Alignment is the process or property of mapping representations from different languages into a shared vector space so that semantically similar words or sentences are close together regardless of language — correcting the natural rotation or mismatch between independent language spaces.

Methods

• Implicit: Multilingual Masked Language Modeling (mBERT) creates implicit alignment.
• Explicit (Supervised): Use parallel corpora (translation pairs) and Minimize MSE($E_{eng}, E_{fr}$) — explicitly pulling translations together.
• TLM (Translation Language Modeling): Perform MLM on concatenated translation pairs, allowing the model to attend from English context to French target.

Why It Matters

• Transfer Success: Better alignment = better cross-lingual transfer.
• Retrieval: Enables Cross-Lingual Information Retrieval (search French docs with English queries).
• Sentence Mining: Used to find parallel sentences in noisy web crawls (like CommonCrawl) to build translation datasets.

Multilingual Alignment is synchronizing the maps — ensuring the vector for "dog" in English lands on top of "perro" in Spanish in the high-dimensional embedding space.