facebookresearch/DensePoseArchived
DensePose
DensePose is a 3D human pose estimation framework designed to map 2D image pixels to a 3D surface-based model of the human body in real time. It functions as a computer vision anatomical mapper that projects 2D visual data onto a 3D surface to create detailed anatomical representations.
The system operates as an image-to-3D texture transfer engine, localizing 2D image annotations onto 3D models to apply photographic textures to digital human representations. It uses a surface-based body mapping method to associate human pixels in an RGB image with specific coordinates on a 3D body template.
The framework covers human pose estimation, 3D human body reconstruction, and real-time motion tracking. These capabilities enable the mapping of human pixels from a 2D image to a 3D surface model to track movement and form.
Features
- 3D Human Mesh Recovery - Implements a framework for reconstructing 3D human bodies and meshes from 2D images in real time.
- 3D Pose Estimation - Analyzes 2D images to determine the precise 3D position and orientation of human body parts.
- Anatomical Mappers - Projects 2D visual data onto a 3D surface to create detailed anatomical representations of people.
- Convolutional Neural Networks - Implements a deep convolutional neural network to extract spatial features from RGB images for 3D coordinate prediction.
- Surface-Based Body Mapping - Associates every human pixel in an RGB image with a specific coordinate on a 3D body template.
- Anatomical Mesh Projections - Projects 2D pixel data onto a predefined 3D human anatomical template for consistent surface representation.
- Texture Transfer Pipelines - Localizes 2D image annotations onto 3D template models to apply photographic textures to digital human representations.
- Image-to-3D Texture Engines - Localizes 2D image annotations onto 3D models to apply photographic textures to digital human representations.
- Video-to-Surface Mapping - Maps human pixels from 2D images onto 3D geometry to track movement and form in real-time.
- Surface Coordinate Mapping - Maps 2D image pixels to a 3D body model using a coordinate system based on surface UV templates.
- Real-Time Motion Tracking - Processes RGB images instantly to map human movement onto a 3D model for interactive applications.
- Per-Pixel Reliability Scoring - Provides per-pixel confidence scores to filter out occlusions and background noise from the 3D mapping.
- Real-Time - Utilizes a high-speed inference runtime for low-latency mapping of 2D image frames to 3D surfaces.
- UV - Predicts exact pixel locations on a 3D mesh by regressing values within a normalized UV map.
- Human Body - Applies visual textures from 2D photographs onto a 3D digital body template for anatomical representations.
- Machine Learning Resources - A framework for mapping pixels to 3D surfaces.
- Pose Estimation Frameworks - Dense human pose estimation mapping pixels to 3D surfaces.
- Real-time Estimation - Dense human pose estimation in the wild.
