Language & LLMs · 2023

Code Llama

Llama learns to program

open source superseded 7B / 13B / 34B / 70B params

Latest: Code Llama 70B (Jan 2024)

Llama 2 specialized for code (Aug 24, 2023): base, Python, and Instruct variants at 7B/13B/34B, with a 70B flagship added January 2024. Supported fill-in-the-middle completion and long 100K-token contexts, becoming the standard open coding model of its era.

Why it matters

Code Llama was the standard open coding model of 2023-24: fill-in-the-middle completion, 100K-token contexts and a 70B Instruct variant that hit 67.8% HumanEval, then the best open result. Released under the Llama 2 license, it became the base for countless open coding assistants before general models absorbed the niche.

Facts

Code Llama 70B Instruct scored 67.8% on HumanEval at release — the best open code model at the time.
Same permissive community license as Llama 2.

Try it yourself

Try it in the Code Llama Playground ↗ Model on Hugging Face ↗ Run it locally with Ollama ↗

Lineage

Descends fromLlama 2

Led toCode World Model Meta LLM Compiler

See the whole family tree →

Sources

arXiv ↗GitHub · codellama ↗Meta AI blog ↗

More in Language & LLMs

LLaMA Llama 3 / 3.1 / 3.2 / 3.3 OPT-175B Galactica Large Concept Models Byte Latent Transformer

Read the Language & LLMs story on the sky →

✦ Open on the map Explore Language & LLMs Quiz me