Open-source alternatives to Hallo2

EchoMimic is a multimodal human animation framework and diffusion-based video generator. It produces lifelike facial and semi-body animations of a reference image by synthesizing motion and appearance from various source data. The system enables portrait animation driven by audio, pose sequences, or driver videos. It features a landmark conditioning tool that allows for the precise control of facial movements by modifying specific landmark points. The framework covers multi-modal motion synthesis and the synchronization of reference images to match the physical movements of a target driver.

AniPortrait is an AI video synthesis pipeline designed to generate photorealistic speaking portraits and facial animations. It functions as a talking head generator and audio-driven animator that synchronizes lip movements, expressions, and head poses to speech or reference video sources. The system includes a facial expression transfer tool for reenacting movements from a source video onto a static reference image. It utilizes a latent diffusion model with reference-based image conditioning to maintain visual identity and consistency across generated frames. The pipeline covers audio-to-exp

Hallo is an audio-driven talking head generator and portrait animation framework. It synchronizes a static portrait image with an audio file to produce realistic talking head videos by mapping audio spectral features to facial expressions and lip movements. The system utilizes a diffusion video synthesis model that employs iterative denoising and latent representations to generate temporally consistent video frames. It incorporates identity-preserving feature extraction and latent space motion modeling to maintain visual consistency and control facial poses. The toolkit provides capabilities

LivePortrait is a deep learning framework for portrait animation that transfers facial expressions from a driving video to a static image. It functions as an AI motion retargeting tool, mapping movements between different identities while preserving the unique features of the source portrait. The system includes specialized capabilities for cross-species portrait animation, adapting human-centric models to non-human subjects and animals. It also features a motion template generator that converts driving videos into portable files to accelerate inference and protect the identity of the origina

This project is a generative adversarial network designed for image animation and motion transfer. It functions as a computer vision framework that synthesizes video sequences by applying motion patterns extracted from a driving video onto a static source image. The model distinguishes itself by using a keypoint-based representation to decouple object appearance from temporal movement. By tracking structural deformations through learned latent coordinates, it performs motion retargeting and synthetic media production without requiring manual annotations or object-specific training data. The

ToonCrafter is a model that combines latent diffusion, reference-based colorization, and sketch-guided control for cartoon animation and interpolation. It functions as a cartoon video interpolation model, a reference-based colorization model, and a sketch-guided animation tool, all built on a latent diffusion animation framework. The project distinguishes itself by integrating three core capabilities into a single pipeline: generating smooth intermediate frames between two cartoon images using diffusion-based priors, transferring color and style from a reference image onto black-and-white ske

This project is a deep learning framework for AI image super-resolution and facial synthesis. It provides a diffusion model image upscaler and a generative facial image synthesizer capable of transforming low-resolution images into high-resolution outputs using pretrained model weights. The system utilizes iterative diffusion refinement and low-resolution guided sampling to restore fine details and sharpness. It supports both unconditional image generation, where images are created from scratch, and guided resolution enhancement for high-fidelity facial reconstruction. The repository include

This project is a Stable Diffusion WebUI extension that provides a graphical interface for personalized portrait generation and AI photo editing. It allows users to train custom identity models from a small set of uploaded images to create consistent digital versions of specific people. The extension includes a virtual try-on system that replaces clothing in images by aligning reference garments with template bodies. It also features tools for face swapping in both static images and videos, as well as a portrait animator that transforms static images into dynamic videos using reference-guided

IOPaint is an AI image editor and Stable Diffusion inpainting tool providing a web interface for removing objects and replacing image content. It utilizes latent diffusion image processing to synthesize high-resolution replacements for erased sections of an image. The project features a specialized AI background remover for isolating subjects and an AI image upscaler that employs super-resolution models for general photos and anime artwork. The software covers a broad range of capabilities including image segmentation for object isolation, face restoration for improving facial details, and t

LatentSync is an audio-driven video generator and latent diffusion lip sync model designed to synchronize a speaker's lip movements in a video to a target audio track. It provides a lip synchronization training framework for developing synchronization networks on custom video and audio datasets. The system utilizes a video preprocessing pipeline to clean, segment, and align face data. It includes a visual sync evaluation tool that calculates confidence scores to measure the accuracy of audio and visual alignment in generated videos. The project covers capabilities for custom synchronization

InfiniteTalk is an open-source system for generating talking head videos driven by audio input. It synthesizes realistic lip movements, head poses, and facial expressions synchronized to a spoken audio track, using either a single still image or a small set of reference video frames as the visual source. The system can produce videos of arbitrary length while maintaining temporal coherence, and it supports animating multiple subjects in a single scene. A key differentiator is the ability to coordinate multiple talking subjects through a structured JSON description, giving each independent lip

RestorePhotos is an AI face restoration tool and deep learning image upscaler designed to remove blur and reconstruct lost details in degraded facial photographs. It functions as a face photo enhancer and a generative adversarial network image processor that transforms low-quality pixels into high-resolution facial features. The system utilizes a GPU-accelerated inference engine to run machine learning models for real-time image restoration. This hardware acceleration supports the heavy matrix multiplications and tensor-based operations required to sharpen facial images and improve visual fid

LiveTalking is an interactive talking head engine and AI avatar management platform designed to synchronize synthetic speech with facial movements. It functions as a real-time orchestrator that connects large language models and text-to-speech services to neural-rendered digital humans. The project distinguishes itself through low-latency streaming capabilities and the ability to handle real-time conversational interruptions. It supports advanced audio-visual customization, including human voice cloning and the ability to drive avatar expressions using real-time webcam data. The platform cov

LivePortrait is a computer vision framework designed for portrait animation and generative video synthesis. It functions as a deep learning system that transfers facial expressions and head movements from a driving video source onto a static image or an existing portrait video, effectively decoupling the subject's identity from the dynamic motion patterns. The framework utilizes keypoint-based motion retargeting and implicit 3D latent representations to map movements across different subjects, including both human and animal portraits. By employing canonical motion normalization and feature-s

This project is a deep learning image matting framework and computer vision tool designed to separate people from their backgrounds. It functions as a real-time video matting engine and a trainable foreground isolation model that generates per-pixel alpha mattes to isolate subjects from photos and videos. The system utilizes reference-based alpha matting, incorporating a specific background image to simulate green screen effects without a physical screen. This approach allows for the removal and replacement of backgrounds in high-resolution footage, including live video streams. The framewor

PaddleGAN is a generative AI framework and deep learning computer vision library built on the PaddlePaddle framework. It serves as a toolkit for image and video synthesis, providing a collection of generative adversarial network implementations for creating synthetic visual content. The library focuses on advanced synthesis capabilities, including the generation of talking heads through lip motion synchronization and the creation of synthetic videos via motion transfer from driving sequences. It provides tools for domain-to-domain translation, allowing for image style transfer and the transfo

GFPGAN is a generative face restoration model and Python-based image processing tool designed to restore low-resolution facial images. It utilizes generative adversarial networks to recover fine details and increase the clarity of degraded portraits. The system employs a generative facial prior to map degraded images to a high-quality manifold, enabling blind-face restoration without requiring knowledge of the specific degradation process. It utilizes a multi-stage workflow that includes face detection, alignment, and region-specific masking to separate facial areas from the background. Beyo

Video-retalking is an AI lip synchronization framework and talking head video editor designed to match the mouth movements of a subject in a video to a target audio track. It utilizes a deep learning pipeline to synchronize speech with video recordings. The system employs a two-stage generation process that separates coarse lip movement from high-resolution detail refinement. It incorporates identity-aware face refinement and expression template alignment to maintain photorealistic skin textures and ensure visual consistency across video frames. The toolset covers facial expression modificat

TecoGAN is a generative adversarial network designed for video super-resolution. It functions as a spatio-temporal video upscaler that increases the resolution of video sequences while reconstructing high-quality imagery from lower-resolution inputs. The system utilizes a temporal coherence framework to ensure visual stability and reduce flickering in generated frames. It achieves this by employing spatio-temporal discriminators that evaluate both individual frame quality and movement consistency. The project covers the training and optimization of generative adversarial networks, specifical

Civitai is a platform for generative media creation and AI model distribution. It provides a centralized service for producing images, videos, audio, and music, while serving as a repository where users can share, discover, and browse custom model weights and fine-tuned adaptations. The platform distinguishes itself through a provider-agnostic orchestration layer that manages multi-step generation pipelines and complex workflows across different backends. It integrates with autonomous AI agents and editors via the Model Context Protocol, allowing external tools to access generation pipelines

DiffBIR is a diffusion-based image restoration framework designed for blind image reconstruction. It utilizes generative diffusion priors to recover high-quality images from sources with unknown or complex degradations without requiring explicit degradation models. The system includes specialized models for face restoration, enabling the recovery of facial landmarks, textures, and backgrounds in degraded portraits. To support high-resolution outputs on hardware with limited memory, it employs a tiled image upscaler that divides images into smaller patches during sampling. The framework cover

This project is a deep learning image restoration tool designed to remove scratches, fading, and noise from aged photographs and film. It utilizes generative adversarial networks for image translation, alongside specialized networks for face enhancement and video colorization. The system distinguishes itself through a combination of latent-space domain mapping and progressive face enhancement to recover blurred or missing high-frequency facial details. For video content, it employs a colorization framework that uses optical flow and temporal guidance to propagate color from selected keyframes

EasyVtuber is 2D avatar animation software that transforms a single static image into a real-time animated character. It functions as a face tracking animation tool and live streaming avatar driver, mapping facial movements from webcams or iOS devices to drive virtual expressions and head motion. The project distinguishes itself through a neural animation pipeline that includes AI video upscaling and frame interpolation to increase visual smoothness and resolution. It utilizes a transparent video streaming system via Spout2, allowing rendered frames with alpha channels to be sent directly to

EchoMimic V2 is an AI video generation pipeline and computer vision animation model designed to produce synthetic human animations. It functions as a generative framework that creates semi-body videos by aligning a static reference image with pose movements extracted from a driving video. The system utilizes a diffusion-based generation process combined with latent space compression and a temporal attention mechanism to ensure smooth transitions between frames. It maintains consistent person identity through reference-based encoding and guides spatial placement via pose-driven motion conditio

BasicSR is a PyTorch-based image restoration toolbox and framework designed for training and deploying deep learning models to upscale, denoise, and deblur images and videos. It serves as a comprehensive system for image super-resolution and video quality restoration, providing the necessary infrastructure to recover fine visual details and increase pixel density. The project distinguishes itself through specialized toolkits for facial image enhancement and high-fidelity face synthesis, as well as a dedicated video quality restoration suite that utilizes deformable convolutions and generative

SadTalker is an audio-driven talking head generator that produces synchronized speaking videos from a single source image and an input audio file. The system utilizes a deep learning framework to map speech signals to facial motion data, enabling the creation of lifelike digital avatars and animated characters. The project distinguishes itself by employing a three-dimensional morphable model to translate audio features into precise facial landmarks and head pose parameters. It integrates latent diffusion motion synthesis to generate naturalistic head movements and uses expression-aware textur

This project is a PyTorch implementation of StyleGAN2, providing a library and research framework for training style-based generative adversarial networks. It serves as a toolkit for high-resolution image synthesis, utilizing competitive minimax optimization to create realistic synthetic visual content. The framework incorporates specialized architectural components such as style-based latent mapping, multi-scale feature modulation, and self-attention layers to improve structural coherence. It distinguishes itself with advanced training stability techniques, including exponential moving avera