PyTorch3D
Fit a 3D mesh from photos by gradient descent
Latest: Actively maintained (PyTorch3D on GitHub)
FAIR's library for deep learning with 3D data: batched meshes and point clouds, differentiable rendering, and loss functions that let gradients flow through the graphics pipeline. Released February 2020, it became the standard toolkit for research at the intersection of vision and graphics, including many of Meta's own 3D papers.
Why it matters
PyTorch3D made 3D differentiable: batched meshes, point clouds and a differentiable renderer that lets gradients flow through the graphics pipeline, so 'fit a mesh from photos by gradient descent' became a homework-sized problem years before NeRF tooling was common. It is the standard toolkit at the vision-graphics intersection and underpins Meta's Codec Avatars research.
Facts
- Its differentiable renderer made 'fit a 3D mesh from photos by gradient descent' a homework-sized problem; used across academia and in Meta's Codec Avatars research.
- Roughly 9k GitHub stars; its differentiable renderer let networks learn 3D shape from 2D photos years before NeRF tooling was commonplace.
Try it yourself
Tutorials on pytorch3d.org ↗ Fit a textured mesh in Colab ↗ Code on GitHub ↗
Lineage
Sources
arXiv ↗GitHub · pytorch3d ↗pytorch3d.org ↗