Multi-view rendering

This project is a PyTorch implementation of a Neural Radiance Field framework. It serves as a 3D scene synthesizer and differentiable volumetric renderer used to train volumetric representations of scenes by predicting color and density for 3D spatial coordinates. The system enables novel view synthesis, allowing for the generation of new images of complex 3D scenes from previously unseen perspectives. It supports 3D scene reconstruction by processing 2D images and camera poses to build a digital volumetric representation of a physical space. The framework includes capabilities for 3D model

PyTorch3D is a 3D geometric deep learning library and mesh processing toolkit designed for learning from point clouds and complex 3D surface geometries. It provides a collection of reusable components and data structures for deep learning with 3D data, including a framework for training and evaluating neural radiance fields to enable photorealistic view synthesis. The project features a differentiable 3D renderer that converts meshes and point clouds into 2D images while allowing gradients to flow back into the geometry and textures. This enables 3D shape optimization, where mesh geometry, te

MultiNeRF is a 3D scene reconstruction suite and framework for training Neural Radiance Fields to synthesize novel views from sets of 2D images. It provides a system for generating new perspectives of a scene by optimizing a neural network based on images and camera poses. The toolkit includes research implementations such as Mip-NeRF 360 and Ref-NeRF for high-fidelity volumetric rendering. It features a structure-from-motion pipeline to calculate camera positions and orientations from image datasets to prepare data for training. The project covers a full workflow for volumetric rendering, i

This project is a framework for neural radiance fields used to synthesize three-dimensional environments from sets of two-dimensional images and camera poses. It functions as a volumetric rendering engine and scene synthesizer that optimizes neural representations of spatial volumes to generate novel views of complex 3D scenes. The system implements a coordinate encoding system that transforms spatial coordinates into high-dimensional space to capture high-frequency geometric details. It also includes a neural mesh extractor that converts trained radiance fields into triangle meshes via march

This project is a computer vision pipeline and volumetric rendering system used to transform photos and videos into high-fidelity 3D models. It implements a deformable neural radiance field framework that optimizes deformation fields to represent non-rigid moving subjects in three dimensions. The system utilizes volumetric deformation fields to map 3D coordinates from a static canonical space to a deformed state. This allows for the reconstruction of photorealistic scenes and the synthesis of high-fidelity images from camera perspectives not present in the original input data. The framework

InstantMesh is a neural 3D reconstruction tool and single-image 3D mesh generator. It utilizes a sparse-view large reconstruction model to convert a single two-dimensional image into a three-dimensional object mesh. The system functions as a textured 3D mesh exporter, saving generated objects with either vertex colors or full texture maps for use in external rendering software. The framework covers a range of capabilities including feed-forward geometry inference, single-image depth estimation, and neural radiance fields. It also supports differentiable mesh rendering and workflows for spars

Neuralangelo is a neural surface reconstruction framework that transforms two-dimensional image sequences and multi-view photography into high-fidelity 3D meshes. It implements a pipeline for training neural radiance fields to represent complex scenes as digital geometry. The project utilizes a signed distance function for surface representation and multi-resolution hash encoding to capture both coarse and fine geometric details. It employs differentiable volume rendering and gradient-based eikonal regularization to ensure the learned distance functions remain physically plausible. The syste

Nerfstudio is a modular development framework for training, visualizing, and exporting three-dimensional scene representations derived from two-dimensional image datasets. It provides a neural scene reconstruction pipeline that converts raw images and camera data into high-fidelity 3D assets and cinematic video using a differentiable volumetric renderer. The system features an interactive web-based visualizer that allows users to monitor training progress and inspect neural scene geometry in real time. It decouples neural network architectures from the training loop through a standardized mod

Threestudio is a 3D generative AI framework designed to create three-dimensional assets from text prompts and images. It provides specialized pipelines for text-to-3D generation and image-to-3D reconstruction, utilizing a neural radiance field trainer to produce geometry and textures. The framework is distinguished by its support for hybrid geometry backends, including signed distance functions, tetrahedra grids, and volume grids. It employs score distillation sampling to guide the generation process and features a modular plugin system for loading custom modules and nodes. The system covers

gsplat is a high-performance differentiable rasterization engine for 3D Gaussian splatting, designed for real-time novel view synthesis from 2D images. It provides a complete pipeline for reconstructing 3D scenes by optimizing differentiable Gaussian representations, training models from COLMAP-processed captures or proprietary device files, and generating new viewpoints through a CUDA-accelerated rendering backend. The framework distinguishes itself through memory-optimized CUDA kernels that reduce training memory usage by up to 4x compared to standard implementations while matching publishe

Mitsuba 3 is a high-performance physically based rendering framework that operates as a CPU and GPU render engine. It functions as a spectral rendering system and a differentiable path tracer, simulating the transport of light as spectral or polarized data through materials and geometry. The system is distinguished by its differentiable rendering pipeline, which calculates derivatives of images relative to input parameters to enable inverse rendering and optimization. It utilizes a just-in-time compilation layer to transform rendering logic into optimized kernels for hardware-agnostic executi

Kaolin is a PyTorch 3D deep learning library providing a comprehensive suite of tools for 3D geometry processing, physics simulation, data visualization, and gradient-based rendering for computer vision. The library includes a differentiable 3D renderer and a geometry processing toolkit for converting and transforming 3D representations such as meshes and point clouds. It also features a 3D physics simulation engine to calculate physical interactions and collisions between three-dimensional objects and scenes. The toolkit provides utilities for 3D data visualization, including the creation o

Neural renderer is a differentiable rendering library for PyTorch that projects three-dimensional meshes into two-dimensional images while maintaining continuous mathematical gradients for backpropagation. The framework enables gradient-based inverse rendering, allowing optimization of input parameters such as camera pose, vertex positions, and texture maps by propagating pixel-level reconstruction errors backward to the source geometry. The architecture incorporates approximate rasterisation gradients that substitute discontinuous edge derivatives with heuristic approximations to facilitate

4DGaussians is a research library and neural rendering engine designed for reconstructing and rendering dynamic three-dimensional scenes. It represents moving environments as a collection of Gaussian primitives that evolve in position and appearance over a temporal dimension. The framework utilizes neural deformation fields to predict spatial offsets and rotations for static point representations, simulating complex motion over time. It further employs temporal basis decomposition to encode motion trajectories into learned functions, compressing dynamic scene data while maintaining smooth tra

SDF-SRN: Learning Signed Distance 3D Object Reconstruction from Static Images 🎯 (NeurIPS 2020)