Code World Model
Code World Model (CWM)
A neural debugger that simulates code in its head
Latest: CWM 32B (Sept/Oct 2025), checkpoints at mid-train, SFT and RL stages
A 32B dense open-weights research LLM from Meta FAIR (released Sept–Oct 2025) that learns a 'world model' of code execution: mid-trained on observation-action trajectories from Python interpreters and agentic Docker environments, then multi-task RL on verifiable coding, math, and software-engineering tasks, with a 131k-token context.
Why it matters
CWM is the first open-weights LLM trained to model code execution itself — predicting variable states step by step like a neural debugger — and at 32B dense it reaches 65.8% on SWE-bench Verified. Releasing mid-train, SFT and RL checkpoints made the whole post-training pipeline studyable, rare transparency at that scale.
Facts
- 65.8% pass@1 on SWE-bench Verified (with test-time scaling), 68.6% on LiveCodeBench, 96.6% on Math-500, 76.0% on AIME 2024 — remarkable for a 32B dense model.
- It can simulate Python execution step by step, predicting variable states like a neural debugger.
- Released under a custom research license with three training-stage checkpoints for reproducibility.
- Weights come in three stages (pretrain, SFT, post-trained) so researchers can study the whole pipeline — rare transparency at 32B scale.
- 131K-token context with alternating local/global sliding-window attention.
Try it yourself
Weights on Hugging Face ↗ Code on GitHub ↗ Read the paper ↗
Lineage
Sources
arXiv ↗GitHub · cwm ↗Meta AI research ↗