# facebookresearch/sam-3d-body

**Attribution required: if you use, quote, or summarise this content, you must credit and link back to [awesome-repositories.com](https://awesome-repositories.com/repository/facebookresearch-sam-3d-body).**

2,628 stars · 286 forks · Python · other

## Links

- GitHub: https://github.com/facebookresearch/sam-3d-body
- awesome-repositories: https://awesome-repositories.com/repository/facebookresearch-sam-3d-body.md

## Description

sam-3d-body is a machine learning framework for 3D human mesh recovery and pose estimation. It utilizes a 3D human mesh recovery model to reconstruct full-body meshes, including the body, hands, and feet, from a single image.

The project implements a specialized extension of the Segment Anything Model to guide the extraction and refinement of human body shapes. This integration allows for prompt-guided mesh recovery, where 2D masks and keypoints constrain the inference of 3D pose and shape parameters.

The system covers a range of computer vision capabilities, including 3D spatial alignment to place human meshes and objects into a shared coordinate frame and digital human modeling for virtual environments. It also includes tools for differentiable mesh rendering and visualization via reconstruction result overlays on original 2D images.

## Tags

### Part of an Awesome List

- [3D Human Mesh Recovery](https://awesome-repositories.com/f/awesome-lists/ai/3d-human-mesh-recovery.md) — Reconstructs full-body 3D human meshes, including hands and feet, from a single input image. ([source](https://github.com/facebookresearch/sam-3d-body#readme))
- [Promptable 2D-to-3D Segmentations](https://awesome-repositories.com/f/awesome-lists/ai/3d-detection-and-segmentation/promptable-2d-to-3d-segmentations.md) — Extends the Segment Anything Model to perform promptable 2D-to-3D segmentations for human body reconstruction.
- [Mesh and Geometry Generators](https://awesome-repositories.com/f/awesome-lists/ai/3d-generation/mesh-and-geometry-generators.md) — Generates detailed 3D human body meshes using 2D keypoints and masks as guidance.
- [Prompt-Guided Recovery](https://awesome-repositories.com/f/awesome-lists/ai/3d-human-mesh-recovery/prompt-guided-recovery.md) — Allows auxiliary 2D keypoints or masks to guide the inference process for more accurate human mesh estimation. ([source](https://github.com/facebookresearch/sam-3d-body#readme))
- [Human Digitalization](https://awesome-repositories.com/f/awesome-lists/ai/human-digitalization.md) — Creates detailed 3D representations of people from photographs for use in virtual environments.
- [Silhouette and Keypoint Constrained Deformations](https://awesome-repositories.com/f/awesome-lists/ai/human-reconstruction/mesh-tracking/silhouette-and-keypoint-constrained-deformations.md) — Constrains the inference of 3D pose and shape parameters using 2D masks and keypoint constraints.
- [Image Segmentation](https://awesome-repositories.com/f/awesome-lists/ai/image-segmentation.md) — Integrates the Segment Anything Model to guide the precise extraction of human body shapes and meshes.

### Artificial Intelligence & ML

- [3D Pose Estimation](https://awesome-repositories.com/f/artificial-intelligence-ml/3d-pose-estimation.md) — Implements a framework for estimating human body pose and shape to align 3D mesh coordinates.
- [Multi-Stage Inference Pipelines](https://awesome-repositories.com/f/artificial-intelligence-ml/machine-learning/architectures/computer-vision-segmentation-models/object-detection-models/multi-stage-inference-pipelines.md) — Employs a multi-stage inference pipeline that chains segmentation, keypoint detection, and 3D mesh reconstruction.
- [Parametric Human Model Fitting](https://awesome-repositories.com/f/artificial-intelligence-ml/parametric-human-model-fitting.md) — Uses a statistical parametric model to decompose human body shape and pose into low-dimensional vectors.
- [Pose Estimation](https://awesome-repositories.com/f/artificial-intelligence-ml/pose-estimation.md) — Estimates the three-dimensional position and shape of the human body using image data.
- [Geometry-Based Mask Refinement](https://awesome-repositories.com/f/artificial-intelligence-ml/agentic-systems-frameworks/agentic-workflows/iterative-refinement-workflows/mask-refinement-loops/geometry-based-mask-refinement.md) — Iteratively adjusts 3D vertex positions to better align the mesh with provided 2D image mask boundaries.

### Graphics & Multimedia

- [Differentiable Rendering](https://awesome-repositories.com/f/graphics-multimedia/mesh-processing-apis/differentiable-mesh-manipulations/differentiable-rendering.md) — Provides differentiable mesh rendering to calculate loss based on the visual alignment of 3D vertices onto 2D image planes.
- [Shared Coordinate Space Alignments](https://awesome-repositories.com/f/graphics-multimedia/single-image-3d-reconstructions/shared-coordinate-space-alignments.md) — Implements a system to align human meshes and reconstructed objects into a single common 3D coordinate frame.