3 个仓库
Awesome GitHub RepositoriesAudio Multi-Conditioning
Architectures that combine diverse inputs like text and melody to guide audio generation.
Distinct from Diffusion Conditioning Architectures: Specializes conditioning architectures for combined text and melodic audio inputs
Explore 3 awesome GitHub repositories matching artificial intelligence & ml · Audio Multi-Conditioning. Refine with filters or upvote what's useful.
Awesome Audio Multi-Conditioning GitHub Repositories
用 AI 发现最棒的仓库。我们将通过 AI 为您搜索最匹配的仓库。
- facebookresearch/audiocraft
facebookresearch/audiocraft
23,379在 GitHub 上查看Audiocraft is a deep learning audio library and machine learning framework designed for training, fine-tuning, and evaluating generative models for music and sound effects. It functions as a text-to-music generative model and a neural audio codec, providing the tools necessary to compress audio signals into discrete representations and synthesize high-fidelity waveforms from textual descriptions. The framework is distinguished by its ability to combine multiple conditioning signals, allowing for the generation of audio based on text prompts, melodic excerpts, or style-based audio clips. It al
Combines textual prompts and melodic excerpts into a shared embedding space to guide the generative process.
Jupyter Notebook - bytedance/latentsync
bytedance/LatentSync
5,806在 GitHub 上查看LatentSync 是一个音频驱动的视频生成器和潜在扩散唇形同步模型，旨在将视频中说话者的唇形动作与目标音轨同步。它提供了一个唇形同步训练框架，用于在自定义视频和音频数据集上开发同步网络。 该系统利用视频预处理流水线来清理、分割和对齐人脸数据。它包括一个视觉同步评估工具，该工具计算置信度分数以衡量生成视频中音频和视觉对齐的准确性。 该项目涵盖了自定义同步网络开发、针对硬件内存和分辨率的训练配置管理以及合成视频评估的功能。
Implements conditioning that injects audio features and facial landmarks to guide the synthesis of synchronized lip movements.
Python - badtobest/echomimic
BadToBest/EchoMimic
4,258在 GitHub 上查看EchoMimic 是一个音频驱动的肖像动画框架和潜在扩散视频生成器。它通过将面部动作与音轨和运动驱动程序同步，将静态参考图像转换为动态的说话头像视频。 该系统作为一个混合运动合成引擎，结合了音频输入和姿势数据。它利用面部标志运动控制器来编辑定位标记，从而实现精确的同步和视频到视频的姿势迁移。 该管道通过潜在扩散和面部标志调节涵盖了图像到视频的动画。这允许由音频、姿势或两种引导源的组合驱动的肖像动画。
Integrates audio signals and visual landmarks to condition the generation of facial movements.
Python
