ImageBind
One embedding space to bind six senses
Latest: ImageBind (May 2023)
The first model to bind six modalities — images, text, audio, depth, thermal, and IMU motion data — into one joint embedding space, trained only on image-paired data (May 2023). Enables cross-modal retrieval and arithmetic, like finding sounds that match an image, without any dataset pairing all modalities together.
Why it matters
ImageBind was the first model to bind six modalities — images, text, audio, depth, thermal and IMU — into one embedding space using only image-paired data, so cross-modal retrieval and embedding arithmetic work between senses that were never paired in training. It prefigured the natively multimodal models Meta later shipped.
Facts
- Embedding arithmetic works across senses: image of a dove + sound of an engine retrieves images of birds near motorbikes.
- Code and weights are open under a non-commercial research license.
Try it yourself
Official ImageBind demo ↗ Code and weights on GitHub ↗ Read the paper ↗
Sources
arXiv ↗GitHub · ImageBind ↗Live demo ↗