Kedreamix/Linly-Dubbing
Linly Dubbing
Linly-Dubbing is an automated video dubbing pipeline designed for multilingual video localization. It converts spoken content in videos into another language by coordinating speech-to-text transcription, text translation, and text-to-speech synthesis.
The system distinguishes itself through AI-driven lip synchronization and animation, which aligns facial expressions and mouth movements to the synthesized voiceover. It also utilizes audio source separation to isolate vocals from background music and noise, allowing for clean voice replacement while preserving original background audio.
The broader capability surface includes tools for web video downloading, timestamped speech transcription, and voice cloning. A graphical configuration interface is provided to manage the processing pipeline, select audio files, and adjust numeric parameters.
Features
- AI Video Dubbing Tools - Coordinates a full pipeline of downloading, transcribing, translating, and synthesizing to produce dubbed videos.
- Lip-Synced - Aligns facial expressions and mouth movements to synthetic audio to produce realistic dubbed video output.
- Audio and Video File Transcription - Transforms spoken audio from video files into timestamped text using automated speech recognition.
- Source Separation Tools - Isolates vocals from background music and noise to ensure clean speech replacement in dubbed content.
- Automatic Speech Recognition - Includes a system for transcribing spoken audio into timestamped text to facilitate translation and subtitle generation.
- Multilingual Voice Cloning Synthesizers - Provides a voice synthesis system that supports voice cloning across multiple languages for video localization.
- Speech Synthesis - Synthesizes natural artificial speech from text with support for voice cloning and multiple languages.
- Speech to Text Transcription - Converts spoken audio into text with precise time markers for subtitle and voiceover alignment.
- Text-to-Speech - Generates artificial human voices from translated text to replace original audio tracks.
- Video Localization Platforms - Localizes video content for international audiences via transcription, translation, and synthetic dubbing.
- Language Translation Services - Provides a modular engine to translate transcribed scripts across multiple languages using swappable services.
- Video Assembly - Combines synthetic voiceovers, original background audio, and subtitles into a final synchronized video file.
- Script Translations - Converts transcribed scripts between languages to prepare content for synthetic audio generation.
- Pipeline Control Panels - Ships a graphical interface for managing the dubbing pipeline, selecting audio files, and adjusting processing parameters.
