wav2vec 2.0
Speech recognition from raw audio, almost no labels
Latest: wav2vec 2.0 (2020); descendants XLS-R, MMS, Omnilingual w2v 7B (2025)
Self-supervised speech representation learning (NeurIPS 2020): the model learns from raw unlabeled audio, then needs astonishingly little labeled data — 10 minutes of transcriptions plus 53K hours of unlabeled speech beat prior systems trained on 100x more labels. The foundation under MMS, Seamless, and 2025's Omnilingual ASR.
Why it matters
Showed speech recognition could learn mostly from unlabeled audio: 53K unlabeled hours plus only 10 minutes of transcripts beat systems trained on 100x more labels. That recipe became the backbone of Meta's MMS, Seamless and Omnilingual ASR, and of most self-supervised speech research since.
Facts
- With just 10 minutes of labeled audio it reached 4.8/8.2 WER on LibriSpeech — a result that reshaped speech research economics.
- Its 2025 descendant, Omnilingual w2v 2.0, scaled the idea to 7B parameters.
Try it yourself
Model on Hugging Face (base, 960h) ↗ Run it with Transformers (docs) ↗ Read the paper ↗