GuyTevet/motion-diffusion-model
Motion Diffusion Model
This is a PyTorch deep learning framework and tool for human motion synthesis that generates 3D character animations from text prompts or action descriptions. It functions as a text-to-motion generator that converts natural language and categorical labels into temporally consistent 3D skeletal movement sequences.
The system utilizes a transformer-based diffusion model to iteratively denoise motion data. It includes capabilities for action-conditioned generation, monocular-to-3D motion lifting, and motion sequence editing using text constraints.
The framework incorporates geometric motion constraint enforcement to ensure physical plausibility through joint-location and velocity losses. It further covers the full animation pipeline, including motion model training, performance evaluation against benchmark datasets, 3D mesh rendering, and physics-based simulation control for environmental interaction.
Features
- Text-to-Motion Generators - Implements a transformer-based diffusion model to generate natural and expressive 3D human animations from text prompts.
- Diffusion Transformers - Implements a transformer-based diffusion architecture to iteratively denoise 3D motion sequences.
- 3D Parametric Human Animation Models - Generates high-quality human movement sequences using 3D parametric models.
- Action-to-Motion Mappings - Implements latent mapping to associate discrete action categories with continuous 3D motion representations.
- Cross-Attention Conditioning - Employs cross-attention mechanisms to condition the diffusion process on natural language embeddings.
- Diffusion Model Frameworks - Ships a PyTorch-based framework specifically for training and sampling from diffusion models for motion.
- Text-to-Motion Generators - Converts natural language prompts into temporally consistent 3D skeletal movement sequences.
- Human Motion Analysis - Provides a toolset for creating and editing physically plausible human animations from learned datasets.
- Iterative Noise Removal - Utilizes an iterative denoising process to refine random noise into structured 3D motion data.
- Human Motion Synthesis - Generates physically plausible human movements conditioned on action labels or monocular data.
- Diffusion Model Training - Provides training workflows for diffusion models to learn mappings between text, actions, and human movement.
- Kinematic Plausibility Constraints - Enforces physical plausibility through joint-location and velocity loss constraints during synthesis.
- 2D-to-3D Mesh Lifting - Translates 2D monocular motion data into 3D skeletal structures using learned diffusion priors.
- Animation Pipelines - Provides a full workflow for generating motion data and rendering 3D meshes for animation software.
- Motion Sequence Editors - Provides capabilities to modify specific parts of movement sequences using text constraints and markers to refine 3D animations.
- Kinematic Constraint Enforcement - Applies losses to joint locations and velocities to ensure generated movements remain physically plausible.
- Motion Generation - Provides a foundational diffusion-based framework for human motion synthesis.
