vt-vl-lab/3d-photo-inpainting

Features

• Inpainting Tools - Implements a system to synthesize missing color and depth data from single RGB-D images to create complete 3D photos.
• Single-Image 3D Reconstructions - Creates a complete 3D digital model from a single photograph by filling hidden or occluded areas.
• Monocular Depth Estimators - Uses pre-trained models to infer three-dimensional spatial depth from a single two-dimensional image.
• Depth Estimation - Provides algorithms for predicting pixel-dense depth information from single input images to guide 3D reconstruction.
• Image Inpainting - Implements generative neural networks to fill missing image regions and restore backgrounds in occluded areas.
• Novel View Synthesis Engines - Generates high-quality novel viewpoints from a single image using multi-layer representations.
• Multi-View Video Synthesis - Generates sequences of images from various camera angles to demonstrate motion parallax effects.
• Novel View Synthesizers - Generates new perspectives of a scene by projecting multi-layer depth images from shifted camera positions.
• Layered Depth Image Processors - Transforms 2D images into multi-layer structures that maintain pixel connectivity for motion parallax effects.
• Layered Depth Image Representations - Transforms single depth-aware images into multi-layer representations to fill occluded areas and enable novel view synthesis.
• 3D Mesh Generators - Converts depth-aware images into inpainted 3D meshes in PLY format for use in graphics engines.
• Connectivity-Based Inpainting - Fills occluded areas by leveraging neighboring pixel data and updating masks of valid regions to ensure consistency.
• Depth Discontinuity Detectors - Identifies and removes edges between pixels with significant disparity differences to detect occlusions and scene boundaries.
• Depth-to-Mesh Generation - Exports depth-aware image data into a standard mesh format for use in graphics engines and 3D software.
• Disparity-Based Discontinuity Detection - Identifies scene boundaries and occlusions by detecting pixels with significant differences in depth values.
• Graph-based Mesh Generation - Converts depth-aware images and camera intrinsics into a graph-based mesh of nodes and edges to model spatial structure.
• Mesh Exporters - Provides utilities for saving generated 3D geometry in standard PLY format for use in graphics engines.
• Image Inpainting Models - Context-aware layered depth inpainting for 3D photography.
• Image inpainting - Listed in the “Image inpainting” section of the Ailia Models awesome list.

Star-Verlauf