3 repositorios
Awesome GitHub RepositoriesAudio Multi-Conditioning
Architectures that combine diverse inputs like text and melody to guide audio generation.
Distinct from Diffusion Conditioning Architectures: Specializes conditioning architectures for combined text and melodic audio inputs
Explore 3 awesome GitHub repositories matching artificial intelligence & ml · Audio Multi-Conditioning. Refine with filters or upvote what's useful.
Awesome Audio Multi-Conditioning GitHub Repositories
Encuentra los mejores repositorios con IA.Buscaremos los repositorios que mejor coincidan usando IA.
- facebookresearch/audiocraft
facebookresearch/audiocraft
23,379Ver en GitHubAudiocraft is a deep learning audio library and machine learning framework designed for training, fine-tuning, and evaluating generative models for music and sound effects. It functions as a text-to-music generative model and a neural audio codec, providing the tools necessary to compress audio signals into discrete representations and synthesize high-fidelity waveforms from textual descriptions. The framework is distinguished by its ability to combine multiple conditioning signals, allowing for the generation of audio based on text prompts, melodic excerpts, or style-based audio clips. It al
Combines textual prompts and melodic excerpts into a shared embedding space to guide the generative process.
Jupyter Notebook - bytedance/latentsync
bytedance/LatentSync
5,806Ver en GitHubLatentSync es un generador de video impulsado por audio y modelo de difusión latente de sincronización labial diseñado para sincronizar los movimientos labiales de un hablante en un video con una pista de audio objetivo. Proporciona un framework de entrenamiento de sincronización labial para desarrollar redes de sincronización en datasets personalizados de video y audio. El sistema utiliza un pipeline de preprocesamiento de video para limpiar, segmentar y alinear datos faciales. Incluye una herramienta de evaluación de sincronización visual que calcula puntuaciones de confianza para medir la precisión de la alineación de audio y video en los videos generados. El proyecto cubre capacidades para el desarrollo de redes de sincronización personalizadas, gestión de configuración de entrenamiento para memoria de hardware y resolución, y evaluación de video sintético.
Implements conditioning that injects audio features and facial landmarks to guide the synthesis of synchronized lip movements.
Python - badtobest/echomimic
BadToBest/EchoMimic
4,258Ver en GitHubEchoMimic is an audio-driven portrait animation framework and latent diffusion video generator. It transforms static reference images into dynamic talking head videos by synchronizing facial movements with audio tracks and motion drivers. The system functions as a hybrid motion synthesis engine that combines audio inputs and pose data. It utilizes a facial landmark motion controller to edit positioning markers, enabling precise synchronization and video-to-video pose transfer. The pipeline covers image-to-video animation through latent diffusion and facial landmark conditioning. This allows
Integrates audio signals and visual landmarks to condition the generation of facial movements.
Python
