Coconut
Coconut (Chain of Continuous Thought)
Reasoning in latent space instead of words
Latest: Coconut (Dec 2024, code released)
FAIR research letting LLMs reason in latent space instead of words: the model's hidden state is fed back as the next input embedding, so 'thoughts' never get flattened into tokens. Coconut can encode multiple candidate next steps simultaneously — an emergent breadth-first search — beating chain-of-thought on logic tasks that require planning and backtracking, with fewer thinking tokens.
Why it matters
Coconut showed LLMs can reason in latent space instead of words: feeding the hidden state back as the next input lets a model hold several candidate reasoning branches at once, an emergent breadth-first search. It beat chain-of-thought on planning-heavy logic tasks with fewer thinking tokens, opening the latent-reasoning research line.
Facts
- Inspired by neuroscience: language areas of the human brain are largely quiet during hard reasoning.
- The continuous thought can hold several possible reasoning branches at once, like superposition.
Try it yourself
Code on GitHub ↗ Read the paper ↗