Projects sharing features with Nerf

This project is a diffusion-based 3D generator and image-to-3D reconstruction system. It translates natural language descriptions or two-dimensional images into three-dimensional assets using neural radiance fields and diffusion models. The system utilizes score-distillation sampling and diffusion-based guidance to refine 3D shapes without requiring 3D training data. It includes specialized tools for transforming neural representations into exportable meshes with texture and material data, as well as a pipeline for iterative optimization of geometry and textures. The project covers a broad r

Nerfstudio is a modular development framework for training, visualizing, and exporting three-dimensional scene representations derived from two-dimensional image datasets. It provides a neural scene reconstruction pipeline that converts raw images and camera data into high-fidelity 3D assets and cinematic video using a differentiable volumetric renderer. The system features an interactive web-based visualizer that allows users to monitor training progress and inspect neural scene geometry in real time. It decouples neural network architectures from the training loop through a standardized mod

This project is a computer vision pipeline and volumetric rendering system used to transform photos and videos into high-fidelity 3D models. It implements a deformable neural radiance field framework that optimizes deformation fields to represent non-rigid moving subjects in three dimensions. The system utilizes volumetric deformation fields to map 3D coordinates from a static canonical space to a deformed state. This allows for the reconstruction of photorealistic scenes and the synthesis of high-fidelity images from camera perspectives not present in the original input data. The framework

Neuralangelo is a neural surface reconstruction framework that transforms two-dimensional image sequences and multi-view photography into high-fidelity 3D meshes. It implements a pipeline for training neural radiance fields to represent complex scenes as digital geometry. The project utilizes a signed distance function for surface representation and multi-resolution hash encoding to capture both coarse and fine geometric details. It employs differentiable volume rendering and gradient-based eikonal regularization to ensure the learned distance functions remain physically plausible. The syste

jetson-inference is a set of libraries and tools for executing optimized deep learning models on embedded GPU hardware. Its primary purpose is to enable real-time computer vision and AI inference at the edge with low latency and high throughput. The project distinguishes itself through high-performance streaming analytics and the ability to execute concurrent AI pipelines on auto-grade silicon. It provides specialized support for multi-sensor stream processing, utilizing zero-copy data transport to load camera frames directly into GPU memory. The codebase covers a broad surface of capabiliti

ml-sharp is a neural radiance field framework designed for single-image 3D reconstruction. It uses a neural network to predict 3D geometry and appearance from a single photograph in a single feedforward pass. The system generates metric 3D scene representations and includes a real-time view synthesizer for producing high-resolution images of new viewpoints. It also features a camera trajectory renderer that creates video sequences by moving a virtual camera through the predicted 3D space. The project covers coordinate-based neural rendering, 3D Gaussian representation regression, and real-ti

Instant-ngp is a high-performance neural graphics engine and toolkit designed for 3D reconstruction and the rendering of neural radiance fields. It provides an integrated framework for generating photorealistic volumetric representations from sets of two-dimensional images by optimizing continuous neural scene models. The project distinguishes itself through a focus on rapid training and real-time inference, achieved by mapping spatial coordinates into compact feature grids. By utilizing multiresolution hash encoding and fused processing kernels, the system minimizes computational overhead an

InstantMesh is a neural 3D reconstruction tool and single-image 3D mesh generator. It utilizes a sparse-view large reconstruction model to convert a single two-dimensional image into a three-dimensional object mesh. The system functions as a textured 3D mesh exporter, saving generated objects with either vertex colors or full texture maps for use in external rendering software. The framework covers a range of capabilities including feed-forward geometry inference, single-image depth estimation, and neural radiance fields. It also supports differentiable mesh rendering and workflows for spars

COLMAP is a 3D scene reconstruction suite and C++ geometry library that implements a full structure-from-motion pipeline. It functions as a GPU-accelerated photogrammetry tool and multi-view stereo framework designed to produce dense 3D geometry and watertight meshes from collections of 2D images. The project distinguishes itself through hardware-accelerated feature extraction and a modular camera modeling system that supports perspective, fisheye, and equirectangular lens types. It employs vocabulary tree image retrieval to efficiently identify similar images in large datasets and provides P

GET3D is a generative 3D mesh model and rendering framework designed to synthesize high-quality textured shapes and tetrahedral meshes. It functions as an image-to-3D reconstructor and text-to-3D generator, utilizing a differentiable 3D renderer to produce realistic visual perspectives and material effects. The system enables the creation of 3D assets from single 2D images, point clouds, or descriptive text prompts. It features a latent space interpolator for creating smooth transitions between different 3D objects and supports the independent control of geometry and texture. The project cov

Wonder3D is a diffusion-based system for single image 3D reconstruction. It generates high-detail 3D meshes from a single input image by producing consistent multi-view normal maps and color images. The pipeline functions as a multi-view normal map generator and a textured mesh extractor. It uses cross-domain multi-view synthesis to create view-dependent maps, which are then converted into 3D geometry through radiance fusion and memory-efficient surface reconstruction. The project covers 3D mesh generation, multi-view generation, and textured 3D modeling. It also includes capabilities for tr

ComfyUI-3D-Pack is a suite of custom nodes for ComfyUI that enables 3D asset generation and rendering within a node-based workflow. It provides a set of tools for reconstructing textured three-dimensional meshes and volumetric scenes from single images, multi-view images, or text prompts. The system includes a Gaussian splatting generator for creating high-fidelity volumetric 3D scene representations and a multi-view image generator to produce consistent image sets for reconstruction. It also features a single image 3D mesh tool to build geometry from a single 2D source. The toolset covers 3

pifuhd is a 3D human reconstruction framework that generates high-resolution 3D meshes of people from a single 2D image. It utilizes pixel-aligned implicit functions to map image pixels to 3D space, predicting surface occupancy and distance to create detailed geometry. The system includes a pipeline for creating digital human assets, moving from 2D image feature projection to the extraction of discrete triangular meshes. It features specialized tools for refining these models, including a post-processor that removes geometric artifacts by isolating the largest connected component of the mesh.

vrn is a 3D face reconstruction tool that generates three-dimensional volumetric representations of human faces from single two-dimensional images. It utilizes a volumetric convolutional neural network regression model to predict 3D volume data directly from image pixels. The system converts these volumetric predictions into 3D meshes through isosurface extraction and vertex coloring. It further applies realistic surface details by mapping two-dimensional image pixels onto the resulting 3D mesh using nearest-neighbor texture projection. The project provides capabilities for single-image dept

This repository contains the code release for Mip-NeRF: A Multiscale Representation for Anti-Aliasing Neural Radiance Fields. This implementation is written in JAX, and is a fork of Google's JaxNeRF implementation. Contact Jon Barron if you encounter any issues.

This repository serves as a comprehensive research platform and toolkit for advancing machine learning, quantum computing, and large-scale scientific data analysis. It provides foundational frameworks for developing complex algorithmic systems, offering the necessary infrastructure for distributed training, computational graph execution, and high-performance model development. The project distinguishes itself by integrating specialized research domains with robust, privacy-preserving methodologies. It supports diverse scientific discovery through tools for quantum simulation, physics-informed

This project is a cross-platform machine learning inference engine designed to execute pre-trained models across diverse operating systems and hardware environments. It functions as a standardized execution framework that manages the entire lifecycle of model inference, from loading and graph optimization to hardware-accelerated execution and generative sequence management. The runtime distinguishes itself through a highly modular architecture that decouples model logic from hardware-specific kernels. By utilizing an execution provider abstraction, it enables developers to offload computation

Oumi is a comprehensive large language model development platform designed for synthesizing data, fine-tuning models, and running performance evaluations. It serves as a unified environment for the entire model lifecycle, encompassing a training and fine-tuning suite, an evaluation framework, and tools for synthetic data generation and model distillation. The platform is distinguished by its iterative, failure-driven synthesis approach, which analyzes model weaknesses during evaluation to generate targeted training data. It utilizes an LLM-based judge framework to programmatically score respo

Threestudio is a 3D generative AI framework designed to create three-dimensional assets from text prompts and images. It provides specialized pipelines for text-to-3D generation and image-to-3D reconstruction, utilizing a neural radiance field trainer to produce geometry and textures. The framework is distinguished by its support for hybrid geometry backends, including signed distance functions, tetrahedra grids, and volume grids. It employs score distillation sampling to guide the generation process and features a modular plugin system for loading custom modules and nodes. The system covers

Gaussian Splatting is a computational framework designed to transform sparse sets of two-dimensional photographs into photorealistic, interactive three-dimensional scene representations. The system functions as a reconstruction tool and rendering engine, enabling the conversion of image data into volumetric models that support novel view synthesis. The project represents scenes as a collection of anisotropic three-dimensional Gaussians, which store position, opacity, color, and covariance data. It distinguishes itself through a differentiable tile-based rasterization process that projects the

gsplat is a high-performance differentiable rasterization engine for 3D Gaussian splatting, designed for real-time novel view synthesis from 2D images. It provides a complete pipeline for reconstructing 3D scenes by optimizing differentiable Gaussian representations, training models from COLMAP-processed captures or proprietary device files, and generating new viewpoints through a CUDA-accelerated rendering backend. The framework distinguishes itself through memory-optimized CUDA kernels that reduce training memory usage by up to 4x compared to standard implementations while matching publishe

Genesis World is an embodied AI simulation platform designed for training robotic agents through physics-based interactions. It centers on a multi-physics simulation engine that integrates rigid body, particle, and finite element method dynamics, supported by a parallel simulation kernel compiler that translates Python functions into optimized GPU and CPU kernels. The platform features a photorealistic robot renderer that utilizes path-tracing and Gaussian Splatting to generate synthetic training data. It includes a domain randomization framework to vary lighting and physical parameters acros

This project is an RGB-D image inpainting tool and framework for 3D photo reconstruction. It transforms single 2D images into 3D content by estimating monocular depth and synthesizing missing color and depth data to fill occluded regions. The system uses a layered depth image representation to manage scene boundaries and pixel connectivity. This allows for novel view synthesis, enabling the generation of videos that simulate motion parallax effects from different camera perspectives. The project covers a range of spatial modeling capabilities, including depth map estimation, disparity-based

4DGaussians is a research library and neural rendering engine designed for reconstructing and rendering dynamic three-dimensional scenes. It represents moving environments as a collection of Gaussian primitives that evolve in position and appearance over a temporal dimension. The framework utilizes neural deformation fields to predict spatial offsets and rotations for static point representations, simulating complex motion over time. It further employs temporal basis decomposition to encode motion trajectories into learned functions, compressing dynamic scene data while maintaining smooth tra

The Book of Shaders is an interactive educational guide and curriculum for learning GLSL fragment shader programming to create procedural graphics and visual effects. It provides a structured learning path and a categorized reference guide for data types, built-in functions, and mathematical operations used in shader development. The project features a web-based shader sandbox and interactive editor that allows for real-time iteration and visualization of GLSL code. Users can experiment with procedural art and share their results via unique URLs. The curriculum covers a wide range of graphic

PyVista is a scientific 3D plotting framework and visualization library that provides a Python interface for rendering and analyzing spatial datasets using a VTK backend. It functions as a volumetric rendering engine and a 3D mesh analysis tool for computing geometric properties and performing boolean operations on surface and volumetric meshes. The project is distinguished by its ability to operate as a headless 3D renderer, generating high-quality renders and animations on remote servers without a physical display. It also features a lazy-accessor extension mechanism that allows the registr

f3d is a fast 3D model viewer and rendering engine designed for visualizing 3D meshes, CAD files, and point clouds. It operates across multiple deployment profiles, functioning as a lightweight desktop application, a scientific data visualizer for volumetric and scalar datasets, a headless rendering engine for automated image generation, and a WebAssembly-based renderer for web applications. The project distinguishes itself through specialized support for Gaussian Splatting scene reconstructions and the ability to visualize complex scientific formats such as VTK, NetCDF, and HDF. It features