Llama 3 / 3.1 / 3.2 / 3.3
405B: the first GPT-4-class model anyone could download
Latest: Llama 3.3 70B (Dec 2024)
The 2024 herd: Llama 3 8B/70B (Apr 2024), Llama 3.1 with the 405B milestone (Jul 2024) — the first open frontier-class model — Llama 3.2 edge sizes (1B/3B) plus 11B/90B vision models (Sep 2024), and Llama 3.3 70B (Dec 2024) matching 405B-level quality at a fraction of the cost.
Why it matters
Llama 3.1 405B was the first openly downloadable model at GPT-4 class, trained on 15T+ tokens across 16,000 H100s, and Llama 3.3 70B then delivered near-405B quality at a fraction of the cost. The 3.2 1B/3B edge models and 11B/90B vision variants made Llama the default open stack from phones to clusters.
Facts
- The first GPT-4-class model anyone could download.
- Llama 3.1 405B was trained on over 15T tokens using more than 16,000 H100 GPUs (~3.8x10^25 FLOPs).
- Llama 3 was trained on two custom 24,576-GPU clusters.
- The 'Herd of Models' paper lists hundreds of contributors.
- 559 listed authors — one of the largest author lists in AI history.
Try it yourself
Model on Hugging Face ↗ Chat with Llama 3.2 in a Hugging Face Space ↗ Run it locally with Ollama ↗
Lineage
Sources
arXiv ↗GitHub · llama-models ↗Hugging Face ↗Meta AI blog · meta llama 3 ↗Meta AI blog · meta llama 3 1 ↗