zju3dv/EasyMocap
EasyMocap
EasyMocap is a markerless 3D human motion capture system that recovers body, hand, and face poses from single or multi-view video without physical markers or suits. It uses parametric body models like SMPL, SMPL-X, and MANO, and leverages mirror reflections to resolve depth ambiguity in single-view pose estimation, improving accuracy by computing mirror surface normals from vanishing points.
The system distinguishes itself through mirror-assisted depth disambiguation, enabling accurate 3D pose reconstruction from a single RGB image or video that includes a mirror reflection. It also supports multi-view triangulation and bundle adjustment calibration for synchronized camera setups, and can fit parametric models to 2D keypoints and silhouettes for robust 3D pose recovery. Reconstructed motion data can be exported to standard animation formats such as BVH and ASF/AMC.
Additional capabilities include CNN-based pose initialization, deformable mesh tracking, and a real-time visualization pipeline for immediate feedback during capture. The project also provides a manual annotation tool for labeling bounding boxes, keypoints, and segmentation masks to create ground-truth data.
Features
- Markerless Motion Capture - Recovering 3D body, hand, and face poses from single or multi-view video without physical markers or suits.
- Multi-View Body-Hand-Face Captures - Recovers body, hand, and face poses from video using parametric models and deformable mesh tracking without physical markers.
- 3D Pose Estimation - Uses a convolutional neural network to produce an initial 3D pose estimate from a single RGB image.
- Mirror-Assisted Depth Disambiguations - Estimating 3D human pose from a single RGB image or video, using mirror reflections to resolve depth ambiguity.
- Mirror-Assisted Reconstructions - Recovers accurate 3D human pose from a single image by leveraging the mirror view to resolve depth ambiguity.
- Multi-View Reconstructions - Reconstructing 3D poses of one or more people from synchronized multi-camera footage using parametric body models.
- Body-Hand-Face Reconstructions - Fits parametric body models (SMPL/SMPL-X/MANO) to synchronized multi-camera footage to extract 3D body, hand, and face poses.
- Mirror-Reflection Depth Resolution - Computes mirror plane normals from vanishing points to resolve depth ambiguity in single-view images.
- Parametric Human Model Fitting - Fits SMPL/SMPL-X/MANO parametric body models to 2D keypoints and silhouettes for 3D pose recovery.
- Stereo Triangulation - Reconstructs 3D joint positions by triangulating 2D detections across synchronized camera views.
- Silhouette and Keypoint Constrained Deformations - Aligns a template mesh to image observations frame-by-frame using silhouette and keypoint constraints.
- Multi-View - Fits SMPL/SMPL-X/MANO body models to synchronized multi-camera footage for 3D human pose reconstruction.
- Camera Calibration - Computes internal camera parameters and the spatial relationship between cameras for multi-view setups.
- Multi-Camera Calibration Systems - Aligns camera perspectives in shared 3D space through intrinsic and extrinsic parameter computation.
- Markerless Reconstructions - Recovers body, hand, and face poses from monocular or multi-view RGB video without markers.
- Mirror Surface Normal Estimations - Computes the orientation of a mirror plane from vanishing points detected in a single image.
- 3D Skeleton and Mesh Overlays - Renders 3D skeleton and mesh overlays on video frames for immediate feedback during capture.
- Animation Data Formats - Exports reconstructed motion data to BVH and ASF/AMC file formats for animation software compatibility.
- Motion Capture Export Formats - Converting captured 3D human motion into standard animation file formats like BVH and ASF/AMC for digital content creation.
- Format Exporters - Converts reconstructed 3D human motion data into standard animation formats like BVH and ASF/AMC.
- Motion Data Export Tools - Converts captured motion data into common animation formats such as BVH and ASF/AMC.
Star history
How this analysis was created: This summary and feature list are AI-generated from collected project material and can contain mistakes. Stars, license and language are imported from GitHub. Inclusion does not mean that we have tested or audited this project. Check the source documentation for any feature you depend on. Learn more on our About page.
