XLM-R
XLM-R (XLM-RoBERTa)
100 languages in one encoder — the ancestor of NLLB
Latest: XLM-R base/large/XL/XXL (2019-2021)
Cross-lingual RoBERTa trained on 2.5TB of filtered CommonCrawl covering 100 languages (November 2019). It delivered massive gains on low-resource languages and set the standard for multilingual understanding — a direct ancestor of Meta's translation moonshots like NLLB.
Why it matters
XLM-R proved one encoder trained on 2.5TB of CommonCrawl across 100 languages could match monolingual models on high-resource languages while lifting low-resource ones dramatically. It became the default multilingual backbone for years — xlm-roberta-base still sees over 20 million monthly Hugging Face downloads — and set up Meta's NLLB translation push.
Facts
- Trained on 2.5 terabytes of text across 100 languages; showed for the first time that one multilingual model could rival monolingual models on high-resource languages while lifting low-resource ones.
Try it yourself
xlm-roberta-base on Hugging Face ↗ xlm-roberta-large on Hugging Face ↗ Read the paper ↗