Perception Encoder & Perception Language Model
A vision encoder whose best features hide in its middle layers
Latest: PE + PLM (Apr 2025)
FAIR's April 2025 open perception stack: Perception Encoder (PE) is a large-scale vision encoder whose intermediate layers surprisingly hold the best embeddings, topping CLIP-style models on image and video tasks; the Perception Language Model (PLM, 1B/3B/8B) is a fully open, reproducible VLM released with PLM-VideoBench for fine-grained video understanding.
Why it matters
Perception Encoder found that a CLIP-style encoder's best embeddings hide in its intermediate layers, not its output, and used that to top CLIP-class models on image and video tasks. PLM paired it with a fully open, non-distilled VLM and 2.5M new human-labeled video QA samples — a deliberate reproducibility stance against proprietary distillation.
Facts
- PLM was trained without distilling from proprietary models — a deliberate 'fully open and reproducible' stance — and shipped with 2.5M new human-labeled video QA samples, then the largest such release.
Try it yourself
PE-Core-G14-448 on Hugging Face ↗ Perception-LM-8B on Hugging Face ↗ Code on GitHub ↗
Sources
GitHub · perception_models ↗Meta AI blog ↗