Meta LLM Compiler
77% of a full autotuning search's gains, without compiling once
Latest: LLM Compiler 7B/13B + 7B-ftd/13B-ftd (Jun 27, 2024)
Foundation models (7B/13B, plus fine-tuned FTD variants) built on Code Llama for compiler optimization: trained on 546B tokens of LLVM-IR and x86/ARM/CUDA assembly to emulate the compiler, tune optimization flags, and disassemble binaries back to IR. Released June 27, 2024 under the bespoke Meta LLM Compiler License permitting both research and commercial use.
Why it matters
The first foundation models for compiler optimization: trained on 546B tokens of LLVM-IR and assembly, the 13B model emulates compiler optimizations 20% of the time versus Code Llama's 0.8%, and the FTD variants improve code size by 4.88% and disassemble with 0.96 round-trip BLEU. Released under a license permitting commercial use.
Facts
- Achieves 77% of the code-size-optimizing potential of a full autotuning search — without running a single compilation.
- Disassembly: 45% round-trip success, 14% exact match, 0.96 round-trip BLEU converting x86_64/ARM assembly back to LLVM-IR.
- FTD-13B beats the compiler's own -Oz flag by 4.88% on code size.
- 16k-token context window; trained on over 500B tokens of compiler IR and assembly.
Try it yourself
LLM Compiler 13B on Hugging Face ↗ LLM Compiler 7B-FTD on Hugging Face ↗ Read the paper ↗
Lineage
Sources
arXiv ↗Hugging Face ↗Meta AI research ↗