xFormers
Exact attention without the O(n²) memory bill
Latest: v0.0.35 (February 2026), with stable wheels for PyTorch 2.10+
Hackable, optimized Transformer building blocks, best known for memory_efficient_attention — exact attention without the O(n²) memory bottleneck, dispatching to the best kernel (including FlashAttention) for the hardware. It became the default speed upgrade for Stable Diffusion and countless training stacks before much of it was absorbed into PyTorch itself.
Why it matters
Made memory-efficient exact attention a one-line drop-in, dispatching to the best kernel for the hardware, including FlashAttention. It was the default speed and memory upgrade for Stable Diffusion and countless training stacks, and much of its work was later absorbed into PyTorch itself.
Facts
- 11k GitHub stars.
- Photoroom measured up to ~100% faster Stable Diffusion image generation just by enabling xFormers' memory-efficient attention.
Try it yourself
pip install xformers ↗ Documentation ↗ Code on GitHub ↗
Lineage
Sources
GitHub · xformers ↗facebookresearch.github.io ↗