Large Concept Models
Large Concept Models (LCM)
An AI that predicts the next idea, not the next word
Latest: LCM 1.6B/7B research release (Dec 2024)
FAIR's bet that the token is the wrong unit of thought: LCMs predict the next sentence-level 'concept' in SONAR embedding space — a language- and modality-agnostic representation covering 200 languages — rather than the next word. Showed strong zero-shot cross-lingual generalization on summarization; training code open-sourced.
Why it matters
LCM challenges the token as the unit of language modeling: it predicts the next sentence-level concept in SONAR's language- and modality-agnostic embedding space covering 200 languages, so one model reasons once and surfaces it in any language. It is FAIR's most explicit architectural bet against the dense next-token transformer.
Facts
- Reasons in an embedding space shared by 200 text languages and speech — so a model 'thinks' once and can surface it in any language.
- The repo picked up ~2,400 stars within a year.
- Because it reasons in a language-agnostic concept space, one trained model generalizes zero-shot to dozens of languages it was never tuned for.
- 21 credited authors ('LCM team').
Try it yourself
Training code on GitHub ↗ pip install sonar-space (the embedding space it reasons in) ↗ Read the paper ↗
Sources
arXiv ↗GitHub · large_concept_model ↗Meta AI research ↗