awesome-repositories.com
Blog
MCP
awesome-repositories.com

Discover the best open-source repositories with AI-powered search.

ExploreCurated searchesOpen-source alternativesSelf-hosted softwareBlogSitemap
ProjectMCP serverAboutHow we rankPress
LegalPrivacyTerms
© 2026 Bringes Technology SRL·VAT RO45896025·hello@awesome-repositories.com
HeartMuLa avatar

HeartMuLa/heartlib

0
View on GitHub↗
3,921 stars·298 forks·Python·apache-2.0·20 views

Heartlib

Heartlib is an audio processing library for large language models that provides tools for audio tokenization, compression, and cross-modal alignment. It implements core models for audio-text embedding, automatic speech recognition, neural codecs, and text-driven audio synthesis.

The project features a text-to-audio synthesis engine capable of generating high-fidelity music and speech from text descriptions or reference files. It also includes a neural audio codec designed for low-bitrate compression that preserves acoustic structure and sound quality.

Additional capabilities cover audio-text alignment via a shared latent space for retrieval, as well as transcription tools specifically designed to convert vocal lyrics and singing into written text.

Features

  • Audio Processing Libraries - A comprehensive library for audio tokenization, compression, and cross-modal alignment for large language models.
  • Audio Tokenization - Converts raw audio signals into discrete numerical representations to preserve acoustic structural patterns.
  • Low-Frame-Rate Tokenizers - Transforms continuous audio streams into discrete, low-frame-rate tokens for efficient data modeling.
  • Neural Audio Compression - Uses neural tokenizers to achieve high-fidelity audio compression at low bitrates.
  • Audio Transcription - Extracts spoken words from music files to convert vocal singing into written text.
  • Automatic Speech Recognition - Provides a pipeline to transcribe vocal lyrics and singing from audio signals into written text.
  • Text-to-Audio Synthesis - Generates high-fidelity audio waveforms from text descriptions or latent representations using a neural synthesis engine.
  • Text-to-Music Engines - Synthesizes high-fidelity audio tracks based on written lyrics, style descriptions, or reference audio.
  • Audio-Text Embedding Models - Implements a shared latent space for mapping music descriptions to audio segments.
  • Autoencoders - Implements autoencoder architectures to compress audio data into latent representations for high-quality reconstruction.
  • Cross-Attention Mechanisms - Implements cross-attention mechanisms to align textual descriptions with corresponding audio segments.
  • Shared Latent Spaces - Maps audio and text into a unified shared latent space to enable cross-modal retrieval.
  • Cross-Modal Audio-Text Alignment - Maps music descriptions and audio segments into a shared space to retrieve matching files.
  • Cross-Modal Retrieval Alignment - Maps music descriptions and audio segments into a shared space for cross-modal retrieval.

Star history

Star history chart for heartmula/heartlibStar history chart for heartmula/heartlib

How this analysis was created: This summary and feature list were written by an AI model that read the project's README and public documentation pages. Each feature links to the documentation it came from; stars, license and language come straight from the GitHub API. The model does not read the source code, and the analysis is refreshed when the project is re-analysed. Learn more on our About page.

AI search

Explore more awesome repositories

Describe what you need in plain English — the AI ranks thousands of curated open-source projects by relevance.

Start searching with AI

Open-source alternatives to Heartlib

Similar open-source projects, ranked by how many features they share with Heartlib.
  • facebookresearch/audiocraftfacebookresearch avatar

    facebookresearch/audiocraft

    23,379View on GitHub↗

    Audiocraft is a deep learning audio library and machine learning framework designed for training, fine-tuning, and evaluating generative models for music and sound effects. It functions as a text-to-music generative model and a neural audio codec, providing the tools necessary to compress audio signals into discrete representations and synthesize high-fidelity waveforms from textual descriptions. The framework is distinguished by its ability to combine multiple conditioning signals, allowing for the generation of audio based on text prompts, melodic excerpts, or style-based audio clips. It al

    Jupyter Notebook
    View on GitHub↗23,379
  • ahmetoner/whisper-asr-webserviceahmetoner avatar

    ahmetoner/whisper-asr-webservice

    3,286View on GitHub↗

    This project provides a self-hosted server for automatic speech recognition, functioning as a containerized inference engine for the Whisper model. It exposes core transcription and translation capabilities through a standardized web interface, allowing for the integration of speech-to-text services into external applications. The service distinguishes itself by incorporating advanced audio analysis tools, including speaker diarization to attribute text to specific individuals and voice activity detection to filter non-speech segments. It supports automated language detection and provides out

    Pythonasrautomatic-speech-recognitiondocker
    View on GitHub↗3,286
  • haoheliu/audioldmhaoheliu avatar

    haoheliu/AudioLDM

    2,830View on GitHub↗

    AudioLDM is a latent diffusion framework for generating high-fidelity audio, music, and sound effects. It functions as a text-to-audio generator that converts natural language descriptions into synthetic audio signals with control over pitch and environment. The system provides specialized tools for audio-to-audio synthesis and generative repair. This includes the ability to perform audio style transfer and replicate specific acoustic events based on existing files. The project covers a broad range of audio transformation tasks, including audio super-resolution for increasing signal fidelity

    Pythonaudio-generation
    View on GitHub↗2,830
  • qwenlm/qwen3-ttsQwenLM avatar

    QwenLM/Qwen3-TTS

    11,976View on GitHub↗

    Qwen3-TTS is a large language model text-to-speech engine designed to convert written text into natural-sounding human speech. It functions as an audio tokenizer and a generative system for speech synthesis. The project features a promptable voice designer for creating synthetic vocal personas based on natural language descriptions. It also includes a zero-shot voice cloning tool that mimics a target speaker using a short reference audio clip and a transcript. The system provides a framework for speech model fine-tuning to improve speaker likeness and quality through supervised training. Add

    Python
    View on GitHub↗11,976
See all 30 alternatives to Heartlib→

Frequently asked questions

What does heartmula/heartlib do?

Heartlib is an audio processing library for large language models that provides tools for audio tokenization, compression, and cross-modal alignment. It implements core models for audio-text embedding, automatic speech recognition, neural codecs, and text-driven audio synthesis.

What are the main features of heartmula/heartlib?

The main features of heartmula/heartlib are: Audio Processing Libraries, Audio Tokenization, Low-Frame-Rate Tokenizers, Neural Audio Compression, Audio Transcription, Automatic Speech Recognition, Text-to-Audio Synthesis, Text-to-Music Engines.

What are some open-source alternatives to heartmula/heartlib?

Open-source alternatives to heartmula/heartlib include: facebookresearch/audiocraft — Audiocraft is a deep learning audio library and machine learning framework designed for training, fine-tuning, and… ahmetoner/whisper-asr-webservice — This project provides a self-hosted server for automatic speech recognition, functioning as a containerized inference… haoheliu/audioldm — AudioLDM is a latent diffusion framework for generating high-fidelity audio, music, and sound effects. It functions as… qwenlm/qwen3-tts — Qwen3-TTS is a large language model text-to-speech engine designed to convert written text into natural-sounding human… quantumnous/new-api — This project is an AI model API gateway and proxy server designed to provide a unified interface for interacting with… openai/openai-go — openai-go is an LLM SDK for Go and a client for interacting with OpenAI services. It provides type-safe bindings to…