KwaiVGI/LivePortrait
LivePortrait
LivePortrait is a deep learning framework for portrait animation that transfers facial expressions from a driving video to a static image. It functions as an AI motion retargeting tool, mapping movements between different identities while preserving the unique features of the source portrait.
The system includes specialized capabilities for cross-species portrait animation, adapting human-centric models to non-human subjects and animals. It also features a motion template generator that converts driving videos into portable files to accelerate inference and protect the identity of the original actor.
The framework covers a range of video editing and animation capabilities, including facial motion retargeting, region-specific masking, and video-to-video expression transfer. It utilizes stitching-based motion transfer and landmark-based retargeting to align driven animations with source images.
Features
- AI Motion Retargeting - Functions as an AI motion retargeting tool that maps facial movements between different identities.
- Portrait Animation Engines - Provides a deep learning engine for transferring facial expressions from driving videos to static portrait images.
- Portrait Animation Tools - Offers tools for generating realistic facial movements in static photos using driving video motion transfer.
- Landmark-Based Retargeting - Adjusts facial landmark scales and positions to accurately map driving expressions onto different identities and species.
- Portrait Video Retargeting - Implements generative retargeting of facial expressions and head movements between different identities.
- Motion Transfer Models - Utilizes a generative motion transfer model to combine driving expression data with static source images.
- Digital Puppet Animation - Creates realistic motion for animal portraits and non-human characters using facial retargeting.
- Animal Motion Synthesis - Applies human facial animations to animal portraits using specialized retargeting and image processing.
- Cross-Species Animation - Provides specialized image processing to adapt human facial animation models to animal subjects.
- Facial Motion Retargeting - Implements precise mapping of facial expressions from a driving video to a target portrait while preserving identity.
- Video Motion Controllers - Transforms existing portrait videos by applying motion and expressions captured from a separate driving clip.
- Region Masking - Isolates targeted portrait areas using masks to refine facial animations without affecting the entire image.
- Regional Editing - Allows for generative modifications and pose adjustments to specific isolated areas of a portrait.
- Motion Template Generators - Converts driving videos into portable motion files to speed up animation and protect the actor's identity.
- Motion Templates - Creates portable motion files to increase processing speed and protect the original actor's identity.
- Template-Based Inference - Reduces redundant calculations and accelerates the animation process using precomputed motion sequences.
- Motion Template Generation - Generates portable motion files to increase processing speed and anonymize the driving actor.
- Cross-Species Animation - Adapts human-centric facial animation models to simulate realistic movements in animal portraits.
- Controllable Generation - Performs efficient portrait animation with stitching and retargeting.
- Image Driven Animation - Efficient portrait animation with precise stitching and retargeting control.
- Generative adversarial networks - Listed in the “Generative adversarial networks” section of the Ailia Models awesome list.
