alievk/avatarify-python
Avatarify Python
Avatarify-python is a real-time face animation tool that uses a PyTorch-based neural network to map facial movements from a live camera feed onto a static image. It creates photorealistic animated avatars that mimic a user's movements for use in video software.
The project includes a remote GPU inference client that offloads heavy computational workloads to a remote server, allowing high-performance animations to run on low-spec hardware. It also features a virtual webcam driver to route synthetic video streams into video conferencing applications as a standard camera device.
The system provides tools for avatar management, including the ability to import custom target images or use generative models to sample synthetic, non-existent human faces. It further implements facial-landmark motion mapping and reference-frame alignment to synchronize the user's facial proportions and pose with the target avatar.
Features
- Real-Time Face Synthesis - Generates photorealistic animated frames by blending target image features with live motion data in real-time.
- Real-Time 2D Avatar Animation - Maps real-time facial movements from a camera feed onto a static image to create a photorealistic animated avatar.
- Facial Animation - Maps real-time facial movements from a camera feed onto a static image to create animated avatars.
- Facial Landmark Analysis - Maps real-time facial landmarks from a live feed to a target image for movement synchronization.
- Remote Inference Offloaders - Offloads heavy neural network inference tasks to a remote GPU server to maintain real-time performance.
- PyTorch Implementations - Implements a deep learning model using PyTorch to animate photorealistic faces based on user movements.
- 2D Avatar Animation Software - Converts static images into dynamic, photorealistic animated avatars using real-time input mapping.
- Virtual Camera Drivers - Provides a virtual camera driver that emulates physical hardware to stream processed video to other apps.
- Remote Processing Offloading - Offloads intensive neural network computations to a remote GPU server to reduce local hardware requirements.
- Virtual Video Device Simulation - Implements a virtual video device to route synthetic avatar streams as a standard webcam feed.
- Realtime Avatar Renderers - Transforms live video feeds into photorealistic animated avatars for real-time streaming.
- Synthetic Face Generators - Provides the ability to sample synthetic, non-existent human faces using generative models for use as avatars.
- Face Alignment - Implements face alignment to ensure user proportions match the target avatar's baseline.
- Pose-Based Animation Alignment - Adjusts camera zoom and reference frames to align the user's face with the target avatar.
- Generative Face Models - Uses generative face models to sample non-existent human faces for use as avatars.
- Synthetic Face Sampling - Samples synthetic, non-existent human faces from generative models to provide a library of avatars.
- Facial Baseline Optimization - Calculates a baseline facial pose manually or automatically to improve animation tracking accuracy.
- Facial Proportion Alignment - Adjusts camera zoom and translation to match the user's facial proportions with a baseline avatar image.
- Virtual Camera Integration - Routes processed animated avatar streams into third-party video conferencing software as a virtual camera input.
