Awesome GitHub RepositoriesPhoneme-Based Alignment

Fairseq is a PyTorch toolkit for sequence-to-sequence modeling, specializing in neural machine translation, automatic speech recognition, and large-scale language model training. It provides a framework for processing and aligning diverse data sources, including text, audio, and video, to support tasks such as speech-to-text conversion and multimodal sequence learning. The project is distinguished by its distributed training capabilities, which utilize parameter sharding, mixed-precision training, and CPU offloading to handle models that exceed single-device memory. It also includes specializ

CosyVoice is a speech synthesis framework that utilizes large language models to generate expressive, multilingual audio. The system functions as an audio generation engine capable of producing natural-sounding speech across multiple languages while preserving regional dialects and specific emotional tones. The platform distinguishes itself through its zero-shot voice cloning capabilities, which allow for the creation of synthetic voice profiles from short audio samples without requiring additional model training. It provides fine-grained control over vocal attributes, enabling users to adjus

WhisperX is an automated speech recognition toolkit designed to convert spoken audio into text while maintaining precise synchronization with the original media. It functions as an integrated pipeline that combines transcription, phoneme-based alignment, and speaker diarization to produce structured, attributed transcripts. The project distinguishes itself through its use of forced alignment, which matches existing text to audio signals at the phoneme level to generate accurate word-level timestamps. It also incorporates speaker diarization to identify and label unique voices within a recordi

Index-tts is a neural audio generation engine designed to convert written text into high-fidelity human speech. By utilizing deep learning models and phoneme-based sequence modeling, the system transforms text into natural-sounding audio waveforms suitable for a variety of accessibility and media applications. The platform functions as a server-side inference pipeline that provides a programmatic interface for integrating voice generation into external applications. It distinguishes itself through asynchronous audio streaming, which buffers and delivers generated speech chunks in real time to

Piper is a local neural text-to-speech engine designed to convert written text into natural human speech entirely on your own hardware. By utilizing a neural synthesis framework, it operates without the need for internet connectivity, ensuring that all audio generation remains private and secure. The system distinguishes itself through a modular architecture that allows for the dynamic loading of speaker embeddings and voice configurations. This enables users to switch between various vocal personas and styles without requiring a full reload of the core synthesis model. By processing input th

VoiceCraft is a neural speech generation and manipulation system consisting of a text-to-speech system, a voice cloning tool, and an audio inpainting engine. It uses a large language model approach to synthesize high-fidelity audio from text and replicate speaker identities. The system provides zero-shot voice cloning and speech editing capabilities, allowing users to modify spoken content within existing recordings. This includes an audio inpainting engine that replaces specific sections of audio with new speech while preserving the original acoustic characteristics and speaker identity. Th

EmotiVoice is an emotional text-to-speech engine and bilingual speech synthesizer designed to generate synthetic audio in English and Chinese. It utilizes a deep learning architecture to produce high-fidelity speech with controllable emotional states and timbres. The project includes a voice cloning framework for replicating specific speaker identities by training custom acoustic models on personal audio datasets. It employs a jointly-trained acoustic-vocoder pipeline and style-embedding-based synthesis to manage expression and reduce audio artifacts. The system covers a broad range of speec

YuE: Open Full-song Music Generation Foundation Model, something similar to Suno.ai but open

Muzic este o platformă și un framework de deep learning pentru analiza, compoziția și sinteza muzicală bazată pe AI. Acesta funcționează ca un framework de generare muzicală și instrument de analiză, utilizând modele de limbaj mari și agenți autonomi pentru a orchestra crearea și interpretarea muzicii simbolice și audio. Proiectul se distinge prin capabilitățile sale cross-modal, mapând limbajul natural și muzica simbolică într-un spațiu de embedding comun pentru clasificare zero-shot și regăsirea informațiilor. Utilizează o varietate de arhitecturi specializate, inclusiv framework-uri de difuzie pentru sinteza audio, mecanisme de atenție dual-grain pentru consistența structurală a secvențelor lungi și un sistem hibrid care combină regulile teoriei muzicale cu rețelele neuronale. Platforma acoperă o gamă largă de capabilități, inclusiv generarea de secvențe MIDI din text și versuri, sinteza vocală neuronală și transcrierea automată a versurilor. De asemenea, oferă instrumente pentru modelarea structurii muzicale, generarea simbolică bazată pe atribute și orchestrarea instrumentelor muzicale externe prin agenți autonomi. Utilitarele de suport includ pipeline-uri de inginerie a datelor pentru binarizarea MIDI la scară largă, codificarea seturilor de date și procesarea semnalului audio pentru extragerea notelor melodice și alinierea vorbirii la foneme.

This project is a TensorFlow voice conversion framework and deep learning audio toolkit designed for neural voice style transfer. It functions as a speech synthesis engine that transforms the spectral characteristics of a source speaker's voice to match the vocal identity of a target speaker. The system employs a phoneme-based approach to voice conversion, classifying audio utterances into speaker-independent phonemes and resynthesizing them using a target voice. This pipeline allows for the transformation of voice characteristics by mapping audio features between different speakers. The too

Voicevox is a text-to-speech synthesis software and audio production environment that converts written text into spoken audio using synthetic character voices. It functions as both a comprehensive editor for voice design and a standalone speech synthesis engine capable of generating audio via an API for integration into external applications. The project distinguishes itself by providing a singing voice synthesizer that uses a piano-roll interface for melodic vocal composition, including the ability to generate humming. It offers specialized prosody editing tools for the manual refinement of