liuzhao1225/YouDub-webui
YouDub Webui
YouDub-webui is a multilingual video translator and AI dubbing pipeline manager featuring a web interface for automating video translation, audio dubbing, and subtitle burning. It utilizes a GPU-accelerated media processor to speed up audio transcription and video rendering tasks.
The system implements a stage-based pipeline that converts original speech into new languages while preserving background audio through audio track mixing. It supports multiple localization workflows, including automated translation and subtitle-driven dubbing using SRT files to bypass automatic transcription phases.
The project includes a task management system for tracking real-time pipeline progress and execution logs. This framework allows users to resume failed localization jobs from the last unsuccessful stage by reusing cached outputs from completed steps.
Users can manage API credentials, network concurrency settings, and external service integrations directly through the web interface.
Features
- Audio-Visual Translation - Provides a comprehensive system for transcribing audio and generating new localized voiceovers while preserving background sounds.
- Video Localization Platforms - Provides an integrated platform for transcribing, translating, and dubbing videos automatically.
- AI Video Dubbing Tools - Implements a workflow to replace original video audio with synthetic translated voiceovers while preserving background sounds.
- Subtitle-Driven Dubbing - Generates dubbed voiceovers and burns subtitles into videos using SRT files instead of automatic transcription.
- Subtitle-Driven Audio Synthesis - Generates dubbed audio by utilizing the timing and content of provided SRT files.
- GPU-Accelerated Inference - Uses GPU-accelerated inference to optimize the speed of audio transcription and voice synthesis models.
- Multi-Platform Video Translation - Processes video localization from multiple sources, including platform links and local file uploads.
- Subtitle-Based Localization - Allows users to localize uploaded videos using provided subtitles to bypass transcription and translation phases.
- Localization Pipeline Stages - Structures the localization process into a sequence of discrete, cacheable steps from transcription to final rendering.
- Audio Mixing - Combines generated synthetic speech with the original background audio to maintain a natural sonic atmosphere.
- Audio Synthesis - Transcribes original speech and generates new translated voiceovers for video localization.
- Hardware-Accelerated Media Processors - Implements a processing engine that leverages GPU acceleration to speed up audio transcription and video rendering tasks.
- Video Translation Pipelines - Manages the sequential pipeline of transcribing, translating, and aligning audio for video localization.
- Intermediate Output Caching - Caches intermediate transcription and translation results to allow tasks to resume without repeating expensive computations.
- Job State Persistence - Persists real-time progress and execution logs in a database to monitor long-running asynchronous localization jobs.
- Task & Job Management - Manages the execution, monitoring, and re-running of video localization tasks.
- Task Pause and Resume Controls - Allows localization jobs to resume from the last successful stage using cached intermediate outputs.
- Task Progress Monitors - Provides utilities for tracking the real-time progress, status, and timing of video localization tasks.
- Real-Time Monitoring Dashboards - Ships a web interface for monitoring the real-time status, logs, and stage-specific durations of the processing pipeline.
