# OpenTalker/SadTalker

**Attribution required: if you use, quote, or summarise this content, you must credit and link back to [awesome-repositories.com](https://awesome-repositories.com/repository/opentalker-sadtalker).**

13,596 stars · 2,607 forks · Python · other

## Links

- GitHub: https://github.com/OpenTalker/SadTalker
- Homepage: https://sadtalker.github.io/
- awesome-repositories: https://awesome-repositories.com/repository/opentalker-sadtalker.md

## Topics

`audio-driven-talking-face` `cvpr2023` `deep-fake` `deep-fakes` `image-animation` `talking-face` `talking-face-generation` `talking-head` `talking-heads`

## Description

SadTalker is an audio-driven talking head generator that produces synchronized speaking videos from a single source image and an input audio file. The system utilizes a deep learning framework to map speech signals to facial motion data, enabling the creation of lifelike digital avatars and animated characters.

The project distinguishes itself by employing a three-dimensional morphable model to translate audio features into precise facial landmarks and head pose parameters. It integrates latent diffusion motion synthesis to generate naturalistic head movements and uses expression-aware texture warping to maintain identity consistency while animating complex facial gestures.

The system covers a broad range of animation capabilities, including the synthesis of rhythmic lip movements and stylized head motions that align with the tone of the provided audio. It incorporates neural rendering and temporal consistency filtering to ensure fluid transitions and high-fidelity visual output across generated video frames.

## Tags

### Artificial Intelligence & ML

- [Talking Head Generators](https://awesome-repositories.com/f/artificial-intelligence-ml/talking-head-generators.md) — Creates realistic speaking videos by synchronizing facial expressions and head movements with input audio.
- [Video Generation](https://awesome-repositories.com/f/artificial-intelligence-ml/video-generation.md) — Generates realistic videos of people speaking by mapping input audio and a single source image to precise facial motion data. ([source](https://sadtalker.github.io))
- [Facial Animation](https://awesome-repositories.com/f/artificial-intelligence-ml/facial-animation.md) — Maps audio signals to source images to produce naturalistic lip-syncing and rhythmic head motion for digital avatars.
- [Facial Landmark Analysis](https://awesome-repositories.com/f/artificial-intelligence-ml/facial-landmark-analysis.md) — Maps audio features to facial landmarks using a three-dimensional morphable model for precise animation control.
- [Audio-Driven Expression Encoders](https://awesome-repositories.com/f/artificial-intelligence-ml/audio-generation-models/expressive-synthesis-models/audio-driven-expression-encoders.md) — Extracts rhythmic and phonetic features from speech to drive the temporal evolution of facial expressions.
- [Latent Diffusion Models](https://awesome-repositories.com/f/artificial-intelligence-ml/generative-ai-resources/diffusion-visual-models/generative-models/latent-diffusion-models.md) — Employs diffusion-based generative models to predict realistic head movement sequences from audio-driven latent representations.
- [Expression-Aware Warping](https://awesome-repositories.com/f/artificial-intelligence-ml/feature-alignment/non-rigid-warping-utilities/expression-aware-warping.md) — Aligns source image features with predicted facial geometry to maintain identity consistency during animation.
- [Generative Adversarial Networks](https://awesome-repositories.com/f/artificial-intelligence-ml/generative-adversarial-networks.md) — Synthesizes high-fidelity video output by training on facial motion patterns and speech audio data using adversarial architectures.
- [Temporal Smoothing Filters](https://awesome-repositories.com/f/artificial-intelligence-ml/sequence-modeling/temporal-sequence-processors/temporal-smoothing-filters.md) — Applies smoothing algorithms across generated video frames to prevent jitter and ensure fluid motion transitions.

### Graphics & Multimedia

- [Portrait Animation Engines](https://awesome-repositories.com/f/graphics-multimedia/media-processing-analysis/face-portrait-manipulation/portrait-animation-engines.md) — Brings static portraits to life with synchronized lip movements and expressive facial gestures driven by spoken audio.
- [Neural Face Renderers](https://awesome-repositories.com/f/graphics-multimedia/real-time-neural-renderers/neural-face-renderers.md) — Synthesizes high-fidelity video frames by projecting learned facial textures onto a geometric mesh using a deep neural network.
- [Human Motion Synthesis](https://awesome-repositories.com/f/graphics-multimedia/animation-motion/animal-motion-synthesis/human-motion-synthesis.md) — Produces varied head movement patterns from audio input to match the rhythm and tone of speech. ([source](https://sadtalker.github.io))

### User Interface & Experience

- [Audio-Driven Animation Engines](https://awesome-repositories.com/f/user-interface-experience/animation-frameworks/state-driven-animations/audio-driven-animation-engines.md) — Automates the synchronization of character head movements and mouth shapes to match the rhythm and tone of provided audio files.
- [Interactive Video Avatar Generators](https://awesome-repositories.com/f/user-interface-experience/avatars/realtime-avatar-renderers/interactive-video-avatar-generators.md) — Generates lifelike talking avatars for virtual presentations by mapping voice data to precise facial motion models.