facebookresearch/map-anything

Features

• 3D Reconstruction - Transforms two-dimensional images into metric three-dimensional spatial representations using feed-forward neural networks.
• Metric 3D Scene Reconstruction - Provides a comprehensive framework for transforming 2D images into metric 3D spatial representations.
• 3D Spatial AI - Transforms two-dimensional images into three-dimensional spatial representations using a feed-forward metric network.
• Metric Coordinate Mapping - Transforms two-dimensional images into three-dimensional spatial representations using a neural network that predicts metric coordinates.
• Model Benchmarking Suites - Implements a standardized interface for evaluating the accuracy of multiple 3D reconstruction models.
• Geometry Model Training - Supports training and fine-tuning of reconstruction models using specialized pose loss functions.
• Reconstruction - Evaluates reconstruction accuracy using standardized datasets across different numbers of input views.
• 3D Reconstruction Benchmarks - Provides a benchmarking suite to standardize and measure the accuracy of diverse 3D reconstruction models.
• Camera Intrinsic Predictions - Recovers camera ray directions and intrinsic parameters from single images without requiring external geometric metadata.
• Ray-Direction Estimations - Recovers camera intrinsic parameters and ray directions from single images without requiring external geometric metadata.
• Camera Geometry Estimation - Estimates camera ray directions and intrinsic parameters from single images without external metadata.
• Vision Dataset Standardizers - Converts diverse spatial datasets into a uniform format to streamline training and cross-model evaluation.
• Image Data Preprocessing - Performs undistortion and depth consistency calculations on image datasets to prepare them for 3D mapping.
• Inference Memory Optimizations - Reduces GPU memory consumption during inference to allow processing of more views on limited hardware.
• Distributed Training - Distributes large-scale geometry model training and fine-tuning workloads across multiple compute nodes.
• Reconstruction Output Standardization - Wraps diverse reconstruction models in a uniform format to align 3D points, camera poses, and confidence scores.
• Third-Party Model Integration - Runs multiple third-party reconstruction models through a single interface to ensure consistent output formats for evaluation.
• Vision Model Fine-Tuning - Optimizes spatial reconstruction models using a modular training pipeline and comprehensive datasets.
• Photogrammetry Pipeline Integrations - Converts neural reconstruction predictions into formats compatible with photogrammetry tools like COLMAP.
• Data Visualization - Renders standardized 3D spatial data to verify the quality of neural reconstruction representations.
• Data Export - Exports neural reconstruction predictions into structured formats compatible with photogrammetry and splatting pipelines.
• Metric Depth Mapping - Processes depth maps and camera poses to maintain geometric consistency in 3D reconstructions.
• Geometric Constraint Integrations - Integrates camera intrinsics, ray directions, and depth maps to improve the accuracy of 3D reconstructions.
• Geometry Artifact Filtering - Denoises and removes edge artifacts from geometry outputs using normal-based masking and depth consistency checks.
• Metric 3D Representations - Renders 3D reconstructed scenes using physically accurate spatial measurements to verify reconstruction quality.
• Photogrammetry Format Exports - Converts reconstruction predictions into files compatible with external photogrammetry and splatting pipelines.
• Geometry Masking - Removes edge artifacts and low-confidence regions by applying depth consistency checks and normal-based filters to outputs.
• Unified Model Interfaces - Wraps reconstruction models in a unified interface to ensure consistent output formats for benchmarking.
• Unified Model Wrappers - Standardizes diverse third-party reconstruction models into a single interface to ensure consistent output formats for benchmarking.
• Reconstruction Benchmarking - Executes standardized evaluation scripts using specific checkpoints and machine configurations to measure project performance.

Star history