Awesome GitHub RepositoriesDeep Learning Audio Libraries

Spleeter is an AI audio source separation library and deep learning toolkit designed to split mixed music files into individual audio stems, such as vocals and drums. It provides a suite of pretrained models for isolating different instruments and voices from a recording. The toolkit includes capabilities for training and evaluating custom audio separation models using labeled datasets and configuration files. It also features utilities for measuring model performance by comparing separation outputs against reference datasets. The system manages audio processing through spectral representati

Audiocraft is a deep learning audio library and machine learning framework designed for training, fine-tuning, and evaluating generative models for music and sound effects. It functions as a text-to-music generative model and a neural audio codec, providing the tools necessary to compress audio signals into discrete representations and synthesize high-fidelity waveforms from textual descriptions. The framework is distinguished by its ability to combine multiple conditioning signals, allowing for the generation of audio based on text prompts, melodic excerpts, or style-based audio clips. It al

EasyEffects is a real-time audio processor and system-wide effects manager designed for PipeWire audio streams. It functions as a comprehensive suite for applying filters, equalizers, and limiters to both input and output audio across the entire system. The project distinguishes itself through its use of deep learning for neural network noise suppression and voice isolation, as well as its ability to simulate physical acoustic environments using impulse-response convolution. It includes a sophisticated preset management system that allows users to associate specific audio configurations with

This project is an end-to-end text-to-speech engine and deep learning voice synthesizer. It functions as a neural speech synthesis framework that converts written text directly into audio waveforms using a single neural network. The system implements an adversarial framework and a conditional variational autoencoder to generate high-fidelity artificial speech. It utilizes a generative adversarial network to ensure synthesized audio is indistinguishable from real human speech. The toolkit provides capabilities for neural speech synthesis, text-to-audio generation, and the training of custom v

该项目是用于原始音频波形生成的 TensorFlow 实现。它作为一个条件语音合成模型，使用扩张卷积神经网络架构产生合成音频样本。 该系统通过在训练和生成过程中结合全局条件和分类标识符来支持自定义语音建模。这使得模型能够模仿特定说话人或为神经文本转语音应用生成独特的音频特征。 该框架涵盖了深度学习音频合成，包括音频数据集处理、从波形文件进行模型训练以及生成可播放的音频文件。它利用扩张因果卷积、mu-law 压扩和量化 Softmax 输出等技术组件来处理音频数据中的长距离依赖关系。

Muzic 是一个用于 AI 驱动的音乐分析、创作和合成的深度学习平台和框架。它作为一个音乐生成框架和分析工具，利用大型语言模型和自主智能体来编排符号音乐和音频音乐的创作与解读。 该项目以其跨模态能力而著称，将自然语言和符号音乐映射到共享的联合嵌入空间中，用于零样本分类和信息检索。它采用了多种专门的架构，包括用于音频合成的扩散框架、用于长序列结构一致性的双粒度注意力机制，以及结合音乐理论规则与神经网络的混合系统。 该平台涵盖了广泛的功能，包括从文本和歌词生成 MIDI 序列、神经歌声合成以及自动歌词转录。它还提供用于音乐结构建模、基于属性的符号生成以及通过自主智能体编排外部音乐工具的工具。 支持性实用程序包括用于大规模 MIDI 二进制化、数据集编码的数据工程流水线，以及用于旋律音符提取和语音到音素对齐的音频信号处理。

Lyra 是一个语音压缩框架和低比特率语音编解码器，旨在通过带宽受限的网络传输高质量音频。它利用自适应比特率音频编解码器在活动会话期间平衡音频质量和网络带宽。 该项目采用生成式音频压缩，使用神经网络从极少量数据中合成语音信号并重建缺失的音频细节。这允许从高度压缩的字节流中重建高质量语音音频。 该系统涵盖了带宽优化的 VoIP 和实时语音通信，专注于低比特率语音压缩以保持通话稳定性。其功能包括动态音频比特率调整和语音音频处理，以防止不稳定网络环境中的延迟和信号丢失。

这是一个基于 TensorFlow 的语音转换框架和深度学习音频工具包，专为神经语音风格迁移而设计。它作为一个语音合成引擎，将源说话人的语音频谱特征转换为目标说话人的声音特征。 该系统采用基于音素的语音转换方法，将音频话语分类为与说话人无关的音素，并使用目标声音重新合成它们。此流水线允许通过映射不同说话人之间的音频特征来转换语音特征。 该工具包包括跨多个 GPU 进行音频模型训练、张量数据归一化以及管理模型超参数的功能。它还提供了用于监控性能的工具，例如通过混淆矩阵可视化分类准确率。

Riffusion-hobby 是一款生成式 AI 工具，通过 Stable Diffusion 生成频谱图图像并将其转换为可播放的音频来创作音乐。它作为一个频谱图音频合成器，利用深度学习将基于图像的声音频率表示转换为音频文件。 该项目作为 AI 音乐推理服务器运行，提供基于 Web 的 API 端点，用于从文本提示词和种子图像生成音频。它还包括用于执行音乐生成任务、配置扩散模型以进行自动化音频创作的命令行界面，以及用于操作声音表示的实时音频生成器。 该系统涵盖了广泛的功能，包括云模型部署、远程推理托管以及用于图像转音频转换的数字信号处理。它还提供了一个交互式 Web 游乐场，用于试验模型参数和探索音乐生成设置。

Stable-audio-tools is a toolkit for training and deploying latent diffusion models for high-fidelity audio synthesis. It provides a framework for generating audio by iteratively refining noise within a compressed latent space, using specialized encoders to preserve temporal and spectral features of the audio signal. The project features a system for adapting pre-trained audio checkpoints to new datasets through modular initialization and configuration files. It includes utilities for weight extraction and inference model export, which remove training metadata and optimizer states to create li

Opus is a lossy audio compression standard and codec designed for high-quality speech and music transmission over the internet. It functions as a low-latency audio codec and network-resilient streamer, providing a framework for encoding and decoding digital audio. The project distinguishes itself through the support of multi-channel ambisonics for immersive three-dimensional spatial audio reproduction. It is specifically optimized for real-time interactive communication, utilizing dynamic bitrate adjustment and forward error correction to maintain audio quality on unstable networks. The syst

该库提供了一个深度学习框架，用于训练神经网络以执行语音识别和音频分类。它利用序列到序列（sequence-to-sequence）架构将变长音频输入映射为文本或数值输出，从而支持自定义语音转录模型的开发。 该项目通过集成的音频处理能力脱颖而出，这些能力将原始波形转换为频谱图和高维数值向量。这些工具允许提取独特的语音特征以识别说话人，以及对特定音频源和口述数字进行分类。 为支持模型开发，该库包含用于音频增强和信号重建的实用程序。通过以编程方式修改音频样本以模拟多样化的声学环境，并验证学习特征的完整性，该系统提高了其底层神经网络的鲁棒性。

This application is a platform for AI voice synthesis and neural voice cloning. It provides a comprehensive toolkit for converting text into natural-sounding human speech by applying custom-trained neural network models to specific audio samples. The system facilitates the entire lifecycle of voice model development, including the preparation of raw audiobooks and video transcriptions into structured training datasets. It supports the training of these models on local or remote hardware, utilizing multi-GPU distributed processing to handle large-scale data and accelerate model convergence. B

This project is a deep learning toolkit designed for audio source separation and music information retrieval. It provides a framework for decomposing polyphonic audio signals into distinct components, such as vocals, drums, and bass, by processing raw waveforms through neural network architectures. The library enables users to train custom separation models or fine-tune existing ones to improve accuracy on specific audio datasets. It supports the entire model lifecycle, including the conversion of raw audio into structured, indexed formats to optimize data loading and training efficiency. Th