# haoheliu/audioldm

**Attribution required: if you use, quote, or summarise this content, you must credit and link back to [awesome-repositories.com](https://awesome-repositories.com/repository/haoheliu-audioldm).**

2,830 stars · 258 forks · Python · other

## Links

- GitHub: https://github.com/haoheliu/AudioLDM
- Homepage: https://audioldm.github.io/
- awesome-repositories: https://awesome-repositories.com/repository/haoheliu-audioldm.md

## Topics

`audio-generation`

## Description

AudioLDM is a latent diffusion framework for generating high-fidelity audio, music, and sound effects. It functions as a text-to-audio generator that converts natural language descriptions into synthetic audio signals with control over pitch and environment.

The system provides specialized tools for audio-to-audio synthesis and generative repair. This includes the ability to perform audio style transfer and replicate specific acoustic events based on existing files.

The project covers a broad range of audio transformation tasks, including audio super-resolution for increasing signal fidelity and audio inpainting for filling missing segments of a recording. These capabilities allow for the restoration and modification of audio signals using text guidance to maintain sonic consistency.

## Tags

### Artificial Intelligence & ML

- [Latent Diffusion Models](https://awesome-repositories.com/f/artificial-intelligence-ml/generative-ai-resources/diffusion-visual-models/generative-models/latent-diffusion-models.md) — Provides a latent diffusion model that performs iterative denoising within a compressed latent space for efficient audio synthesis.
- [Audio Generation Models](https://awesome-repositories.com/f/artificial-intelligence-ml/audio-generation-models.md) — Implements a specialized audio generation model for producing high-fidelity sound effects, music, and speech.
- [Cross-Attention Mechanisms](https://awesome-repositories.com/f/artificial-intelligence-ml/generative-ai-resources/diffusion-visual-models/generative-ai-architectures/cross-attention-mechanisms.md) — Uses cross-attention mechanisms to integrate text embeddings into the diffusion process for guided audio generation.
- [Spectrogram Encoders](https://awesome-repositories.com/f/artificial-intelligence-ml/generative-ai-resources/diffusion-visual-models/generative-ai-models/latent-space-generative-models/latent-space-projections/latent-space-encoders/spectrogram-encoders.md) — Transforms audio into spectrogram representations before encoding them into the latent space for processing.
- [Variational Autoencoders](https://awesome-repositories.com/f/artificial-intelligence-ml/model-training/variational-autoencoders.md) — Utilizes a variational autoencoder to compress raw audio waveforms into continuous latent distributions for generative tasks.
- [Iterative Denoising Pipelines](https://awesome-repositories.com/f/artificial-intelligence-ml/neural-network-architectures/u-net-architectures/iterative-denoising-pipelines.md) — Employs an iterative denoising pipeline to recover high-fidelity audio signals from Gaussian noise.
- [Text-to-Audio Synthesis](https://awesome-repositories.com/f/artificial-intelligence-ml/text-to-audio-synthesis.md) — Functions as a neural text-to-audio synthesis system converting natural language descriptions into speech, music, and sound effects. ([source](https://cdn.jsdelivr.net/gh/haoheliu/audioldm@main/README.md))
- [Audio-to-Audio Synthesis](https://awesome-repositories.com/f/artificial-intelligence-ml/audio-to-audio-synthesis.md) — Creates new audio clips based on existing files to replicate or transform specific acoustic events. ([source](https://cdn.jsdelivr.net/gh/haoheliu/audioldm@main/README.md))
- [Audio Style Transfers](https://awesome-repositories.com/f/artificial-intelligence-ml/image-generation/audio-style-transfers.md) — Modifies the style of sound files using text prompts while preserving the original audio structure. ([source](https://cdn.jsdelivr.net/gh/haoheliu/audioldm@main/README.md))
- [Audio Inpainting And Editing](https://awesome-repositories.com/f/artificial-intelligence-ml/text-to-audio-synthesis/audio-inpainting-and-editing.md) — Enables surgical audio modification and segment regeneration using text-based guidance to fill recording gaps. ([source](https://audioldm.github.io/))

### Graphics & Multimedia

- [Audio Super-Resolution](https://awesome-repositories.com/f/graphics-multimedia/audio-music/audio-processing/audio-super-resolution.md) — Provides audio super-resolution capabilities to increase the sample rate and fidelity of low-quality recordings. ([source](https://audioldm.github.io/))