nerfstudio-project/gsplat
Gsplat
gsplat is a high-performance differentiable rasterization engine for 3D Gaussian splatting, designed for real-time novel view synthesis from 2D images. It provides a complete pipeline for reconstructing 3D scenes by optimizing differentiable Gaussian representations, training models from COLMAP-processed captures or proprietary device files, and generating new viewpoints through a CUDA-accelerated rendering backend.
The framework distinguishes itself through memory-optimized CUDA kernels that reduce training memory usage by up to 4x compared to standard implementations while matching published quality metrics. It supports large-scale scene reconstruction with millions of Gaussians through gradient-based densification and pruning strategies, multi-GPU distributed rendering for handling massive scenes, and antialiased rendering to minimize visual artifacts. Additional capabilities include rendering arbitrary-dimensional feature vectors beyond RGB, compressing scene representations for reduced storage, and providing a browser-based interactive viewer for real-time exploration of trained models.
The project covers the full workflow from scene optimization and rendering to performance profiling and quality evaluation, with automated benchmark execution for reproducing standard metrics on common datasets.
Features
- Gaussian Splatting - Provides a high-performance differentiable rasterization engine for real-time novel view synthesis from 3D Gaussian splats.
- Gaussian Splatting Optimizers - Optimizes 3D Gaussian scene parameters by backpropagating gradients from rendered 2D images through a differentiable pipeline.
- Differentiable 2D Projections - Projects 3D Gaussian primitives onto a 2D image plane using a differentiable rasterization pipeline for real-time novel view synthesis.
- Rasterization Accelerators - Renders 3D Gaussian splats into 2D images using custom CUDA kernels with automatic gradient computation.
- Gaussian Splatting Rasterizers - Provides a CUDA-accelerated rasterization pipeline that processes millions of 3D Gaussian primitives in real time.
- Novel View Synthesis Engines - Generates novel viewpoints from sparse 2D images using optimized 3D Gaussian representations.
- Gaussian Splatting View Synthesizers - Generates novel viewpoints from 2D images by optimizing and rendering a 3D Gaussian representation through differentiable rasterization.
- Gaussian Scene Optimizers - Optimizes 3D Gaussian parameters by backpropagating gradients through a differentiable rasterization pipeline.
- Gaussian Splatting Optimizers - Provides a training pipeline that backpropagates gradients from rendered images to refine 3D Gaussian parameters.
- CUDA Kernel Optimizations - Ships custom CUDA kernels that cut training memory usage by up to 4x for 3D Gaussian splatting models.
- Accelerated Gaussian Splatting Trainers - Trains 3D Gaussian splatting models using up to 4x less GPU memory and 15% less time than the official implementation.
- Gaussian Splatting Reconstructors - Reconstructs scenes with millions of 3D Gaussian primitives using gradient-based densification and pruning strategies.
- Differentiable Geometry Renderers - Renders 3D Gaussian splats into 2D images using a differentiable pipeline for gradient-based optimization.
- 3D Scene Renderers - Renders large 3D scenes in real time using CUDA-accelerated differentiable rasterization of Gaussians.
- Gaussian Splatting Tile Sorters - Implements tile-based sorting of 3D Gaussians for efficient real-time rasterization in depth order.
- Memory-Optimized CUDA Kernels - Ships memory-optimized CUDA kernels that cut training memory usage by up to 4x versus standard implementations.
- Gaussian Densification - Implements gradient-based densification that adds Gaussians in under-reconstructed regions during scene optimization.
- Gradient-Based Densification Strategies - Implements gradient-based densification to add and prune Gaussians in under-reconstructed regions during training.
- Gaussian Splatting Multi-GPU Distributions - Splits 3D Gaussian rasterization across multiple GPUs for larger scenes and faster throughput.
- Antialiasing - Applies a low-pass filter during rasterization to minimize aliasing artifacts in novel view synthesis.
- Feature Vector Renderers - Renders arbitrary-dimensional feature vectors attached to 3D Gaussians beyond standard RGB.
- Anti-Aliasing Techniques - Applies antialiasing techniques during rendering to minimize visual artifacts in novel view synthesis.
- Multi-GPU Gaussian Splatting Renderers - Provides multi-GPU distributed rendering to handle massive 3D Gaussian scenes and increase throughput.
- Gradient-Based Cloning - Duplicates Gaussians in high-gradient regions to focus computational capacity on under-reconstructed scene areas.
- Opacity Reset - Periodically resets Gaussian opacity values to prevent permanent transparency during scene optimization.
- Extremely Large Gaussian Scene Renderers - Processes extremely large 3D Gaussian scenes faster than standard CUDA backends.
- Pruned Gaussian Scene Renderers - Renders millions of Gaussians in real time by discarding distant ones based on projected radius.
- Gaussian Splatting Viewers - Ships a browser-based interactive viewer for real-time exploration of trained 3D Gaussian models.
- Gaussian Splatting Viewers - Ships a browser-based viewer for real-time exploration of trained 3D Gaussian models.
- Gaussian Pruning - Prunes redundant Gaussians that contribute negligibly to rendered images, reducing memory and computation.
- Single-Image 3D Reconstructions - Optimizes 3D Gaussians to reconstruct a full 3D scene from a single 2D input view.
- Gaussian Fitting Reconstructions - Ships a single-image reconstruction demo that fits 3D Gaussians to a single training image.
- Gaussian Feature Vector Renderers - Renders multi-dimensional feature vectors attached to 3D Gaussians beyond standard RGB.
- Gaussian Splitting - Splits oversized Gaussians into smaller ones to capture fine details during scene optimization.
- Densification Strategy Application - Applies absgrad and MCMC-based densification strategies to manage Gaussian count during scene optimization.
- Core Implementations - Integration of Gaussian splatting within the Nerfstudio ecosystem.
- Gaussian Splatting - CUDA-accelerated rasterization library for Gaussian splatting.
