facebookresearch/pytorch3d

Features

• Differentiable Geometry Renderers - Generates 2D images from 3D meshes or point clouds while allowing gradients to flow back into the geometry and textures.
• Differentiable Rasterizers - Provides a rendering system that projects 3D primitives into image space while supporting gradient flow for optimization.
• 3D Machine Learning Libraries - Provides a comprehensive set of reusable components and data structures for deep learning on 3D spatial datasets.
• Coordinate-Based Neural Representations - Implements continuous functions that map spatial coordinates to color and density values for photorealistic view synthesis.
• Geometric Deep Learning Frameworks - Provides specialized tensors and operations for applying deep learning to non-Euclidean data such as point clouds and meshes.
• Heterogeneous 3D Batching - Processes groups of 3D objects with varying vertex or face counts within a single GPU-accelerated tensor operation.
• Heterogeneous Batching - Groups 3D objects with varying vertex or face counts into single tensors using offsets for GPU acceleration.
• Novel View Synthesis Engines - Produces novel viewpoints of a scene using implicit representations of 3D geometry and appearance.
• Neural Radiance Field Synthesizers - Optimizes neural representations of spatial volumes to synthesize novel views of 3D scenes.
• Radiance Field Training Pipelines - Optimizes coordinate-based neural networks to represent 3D scenes from a set of 2D images.
• Heterogeneous Batching - Processes groups of 3D objects with varying vertex or face counts within a single tensor operation.
• Differentiable Shape Optimization - Refines 3D mesh geometry and textures to match target images through a differentiable optimization pipeline.
• 3D Scene Renderers - Generates 2D images from 3D meshes or point clouds using a differentiable pipeline that allows gradients to flow back into geometry.
• Mesh Processing APIs - Provides programmatic interfaces for processing and transforming triangle meshes and point clouds using tensor operations.
• Differentiable Mesh Manipulations - Performs operations on triangle meshes including projective transformations and sampling using differentiable tensors.
• Mesh Processing Tools - Offers a toolkit for importing, manipulating, and transforming triangulated mesh data and point clouds.
• Triangle Mesh Data Structures - Stores and manipulates triangle meshes using specialized data structures and efficient projective transformations.
• Textured Mesh Rendering - Generates 2D images from 3D mesh data by simulating light interaction and camera parameters.
• Geometric Processing Kernels - Utilizes high-performance CUDA kernels optimized for geometric transformations and sampling on spatial data.
• Deep Learning Components - Provides reusable building blocks for organizing and managing 3D data processing and learning components.
• Training Execution Loops - Provides managed training execution loops for optimizing scene-specific or generalizable 3D neural representations.
• Mesh Deformations - Transforms a base 3D mesh into a target shape by optimizing vertex positions to match specific geometry.
• Textured Mesh Optimizations - Optimizes a 3D mesh and its associated texture to match a target image using a differentiable rendering pipeline.
• Custom Shader Programs - Combines texturing, lighting models, and blending methods through user-defined shader logic.
• Modular Shading Pipelines - Separates geometry projection from lighting and blending logic to enable flexible definition of surface appearances.
• Extensible Rendering Pipelines - Provides a versatile graphics architecture that supports batching and gradients within a modular pipeline.
• Texture Mapping Pipelines - Maps colors to mesh surfaces using vertex interpolation, UV texture maps, or per-face texture atlases.
• Spatial Point Cloud Renderers - Processes large-scale point clouds using optimized kernels for high-resolution output.
• Volumetric Representation Optimizations - Optimizes a 3D volumetric representation to match target images by refining volume density and color.
• Camera Geometry Estimation - Implements algorithms to refine 3D camera positions and orientations by minimizing reprojection errors.
• Computer Vision - Library for 3D computer vision and deep learning.
• Neural Radiance Field Implementations - Library for 3D deep learning and differentiable rendering operations.
• Neural Radiance Fields - Library for deep learning with 3D data structures and rendering.
• Processing Libraries - Deep learning library for 3D data processing and research.

Star history