SAM 2
One click on one frame tracks an object through a whole video
Latest: SAM 2.1 (Sep 2024), plus the SAM 2.1 Developer Suite with open training code
Extends promptable segmentation to video with a streaming-memory transformer: prompt an object once and SAM 2 tracks its 'masklet' across frames in real time (~44 fps), surviving occlusions and reappearances. Also 6x faster and more accurate than SAM on still images. Apache 2.0 code and weights, July 2024.
Why it matters
SAM 2 extended promptable segmentation to video: one click on one frame tracks an object through occlusions at about 44 fps, and it is 6x faster than SAM on stills. Its SA-V dataset (51,000 videos, 643,000 masklets) was roughly 50x larger than prior video segmentation data, making it the standard video-mask tool.
Facts
- It tracks an object through an entire video from a single click on one frame.
- Its SA-V dataset (about 51,000 videos, 643,000 masklets, CC-BY-4.0) is roughly 50x larger than prior video segmentation datasets.
- Using SAM 2 in the annotation loop made labeling 8.4x faster than per-frame manual annotation.
- SA-V is 53x larger than the previous biggest video object segmentation dataset; masklets are 70% auto-generated (451.7K) plus 190.9K manual, filmed across 47 countries with 14-second average clips.
- 18 authors (Ravi et al.).
Try it yourself
Official SAM 2 demo ↗ sam2.1-hiera-large on Hugging Face ↗ Run the official notebook in Colab ↗
Lineage
Sources
arXiv ↗GitHub · sam2 ↗Meta AI blog ↗Meta AI ↗