# facebookresearch/sapiens

**Attribution required: if you use, quote, or summarise this content, you must credit and link back to [awesome-repositories.com](https://awesome-repositories.com/repository/facebookresearch-sapiens).**

_How this analysis was created: the description and tags below were written by an AI model that read this project's README and public documentation pages; stars, license and language come straight from the GitHub API. The model does not read the source code._

5,388 stars · 319 forks · Python · NOASSERTION

## Links

- GitHub: https://github.com/facebookresearch/sapiens
- Homepage: https://about.meta.com/realitylabs/codecavatars/sapiens/
- awesome-repositories: https://awesome-repositories.com/repository/facebookresearch-sapiens.md

## Description

Sapiens is a high-resolution human vision model designed for high-precision, human-centric computer vision tasks. It functions as a suite of tools for estimating human pose, depth, and surface geometry.

The project utilizes a vision transformer backbone to perform multiple tasks through a shared encoder. This architecture enables the simultaneous prediction of skeletal structures, joint locations, and the distance between a camera and a human subject.

The model's capabilities cover human body part segmentation to isolate anatomical regions from backgrounds and surface normal prediction to recover 3D geometric details from 2D images. These tasks are supported by a multi-task learning framework that employs pixel-wise regression and semantic segmentation masking.

## Tags

### Part of an Awesome List

- [Human Pose Estimation](https://awesome-repositories.com/f/awesome-lists/ai/pose-estimation-frameworks/human-pose-estimation.md) — Predicts human anatomical keypoints and skeletal structures from 2D images. ([source](https://cdn.jsdelivr.net/gh/facebookresearch/sapiens@main/README.md))
- [Human Centric Perception](https://awesome-repositories.com/f/awesome-lists/ai/human-centric-perception.md) — Provides a high-resolution vision model for multi-modal human-centric tasks including pose, depth, and geometry.
- [Surface Normal Estimation](https://awesome-repositories.com/f/awesome-lists/ai/vanishing-point-detection/surface-normal-estimation.md) — Predicts the orientation of body surfaces to capture detailed structural information from images. ([source](https://about.meta.com/realitylabs/codecavatars/sapiens/))

### Artificial Intelligence & ML

- [Human](https://awesome-repositories.com/f/artificial-intelligence-ml/artificial-intelligence-tooling/language-model-integrations/computer-vision-models/pose-estimation-models/human.md) — Delineates human anatomical regions and joint locations to separate people from backgrounds.
- [Depth Estimation](https://awesome-repositories.com/f/artificial-intelligence-ml/computer-vision-systems/computer-vision/object-pose-estimations/monocular-depth-estimators/multi-view-depth-estimators/depth-estimation.md) — Calculates the distance between the camera and humans to create spatial scene representations. ([source](https://cdn.jsdelivr.net/gh/facebookresearch/sapiens@main/README.md))
- [Human Body Part Segmentation](https://awesome-repositories.com/f/artificial-intelligence-ml/human-body-part-segmentation.md) — Provides high-resolution segmentation to isolate anatomical regions of the human body from backgrounds.
- [Human-Centric Vision Systems](https://awesome-repositories.com/f/artificial-intelligence-ml/human-centric-vision-systems.md) — Performs high-precision vision tasks like pose and depth using a shared Vision Transformer backbone.
- [Human Depth Estimation](https://awesome-repositories.com/f/artificial-intelligence-ml/human-depth-estimation.md) — Calculates the distance between a camera and a human subject to create spatial representations.
- [Human Surface Normal Prediction](https://awesome-repositories.com/f/artificial-intelligence-ml/human-surface-normal-prediction.md) — Determines the orientation of body surfaces to recover 3D geometric details from 2D images.
- [Keypoint Detection](https://awesome-repositories.com/f/artificial-intelligence-ml/keypoint-detection.md) — Implements anatomical landmark identification for human joints using heatmap-based probability distributions.
- [Multi-Task Learning Models](https://awesome-repositories.com/f/artificial-intelligence-ml/machine-learning/architectures/sequence-models/multi-task-learning-models.md) — Employs a shared encoder architecture to simultaneously perform depth, pose, and segmentation tasks.
- [Shared-Bottom Architectures](https://awesome-repositories.com/f/artificial-intelligence-ml/machine-learning/architectures/sequence-models/multi-task-learning-models/shared-bottom-architectures.md) — Uses a shared-bottom architecture to extract features for multiple specialized human-centric output heads.
- [Surface Normal Estimators](https://awesome-repositories.com/f/artificial-intelligence-ml/surface-normal-estimators.md) — Determines the orientation of body surfaces to recover 3D geometric details from 2D images. ([source](https://cdn.jsdelivr.net/gh/facebookresearch/sapiens@main/README.md))
- [Vision Transformers](https://awesome-repositories.com/f/artificial-intelligence-ml/vision-transformers.md) — Uses a transformer-based backbone to process images as patches for high-resolution feature extraction.
- [Semantic Segmentation Architectures](https://awesome-repositories.com/f/artificial-intelligence-ml/vision-transformers/encoder-decoder-architectures/semantic-segmentation-architectures.md) — Uses neural network structures to produce pixel-wise semantic labels for isolating human subjects.
- [Feature Map Upsamplers](https://awesome-repositories.com/f/artificial-intelligence-ml/feature-alignment/feature-map-upsamplers.md) — Utilizes decoder layers to increase feature map resolution for precise pixel-level predictions.
- [Pixel-Wise Reconstruction Losses](https://awesome-repositories.com/f/artificial-intelligence-ml/machine-learning/architectures/neural-network-components/loss-functions/perceptual-loss/content-loss-calculators/focal-loss-calculators/detection-loss-calculators/smooth-l1-loss-calculators/regression-loss-functions/pixel-wise-reconstruction-losses.md) — Predicts continuous depth and surface normal values by minimizing pixel-wise reconstruction error.

### Data & Databases

- [Semantic Part Segmentation](https://awesome-repositories.com/f/data-databases/multi-part-upload-orchestration/3d-model-part-splitting/semantic-part-segmentation.md) — Isolates specific anatomical regions using deep learning to partition the human subject.

### Graphics & Multimedia

- [Human Body Part Segmenters](https://awesome-repositories.com/f/graphics-multimedia/background-segmentation/human-body-part-segmenters.md) — Identifies and delineates specific anatomical regions to separate human subjects from backgrounds. ([source](https://cdn.jsdelivr.net/gh/facebookresearch/sapiens@main/README.md))