mkocabas/VIBE
VIBE
VIBE is a 3D human pose estimation framework designed to reconstruct human body shapes and poses from video frames. It functions as a toolkit for predicting parameters of the SMPL human body model to generate 3D mesh sequences.
The system includes a 3D motion data exporter to convert predicted pose sequences into standard 3D file formats for use in graphics and animation software. It also provides a structured training pipeline for preparing datasets and training models to estimate body shapes from images.
Its capabilities cover computer vision for estimating body pose and shape, as well as machine learning operations for model training. The framework supports multi-person sequence detection and parametric shape optimization to align 3D meshes with 2D detections.
Features
- Pose Estimation Frameworks - Provides a comprehensive framework for predicting body model parameters to reconstruct the physical pose and shape of multiple people.
- 3D Pose Estimation - Converts 2D video detections into three-dimensional spatial coordinates to reconstruct human body movement.
- Body-Hand-Face Reconstructions - Fits the SMPL parametric body model to video footage to extract 3D body poses and shapes.
- Parametric Human Model Fitting - Optimizes parameters of pre-defined human body meshes to align 3D reconstructions with 2D detections.
- Video Pose Estimation - Implements techniques for tracking and predicting human motion across temporal video sequences to create continuous trajectories.
- 3D Pose Model Training - Includes a pipeline for preparing datasets and training neural networks specifically for 3D joint and shape estimation.
- Multi-Person Trackers - Identifies and tracks multiple distinct human figures across video frames to estimate independent body poses.
- Format Exporters - Converts predicted pose sequences into standard 3D animation file formats for use in external graphics software.
- Motion Data Export Tools - Provides tools to export reconstructed 3D human motion data into standard animation file formats for use in graphics software.
- 3D Human Mesh Recovery - Video-based human mesh recovery from monocular video.
