# Speech processing library

> AI-ranked search results for `speech processing` on awesome-repositories.com — ordered by an LLM for relevance, best match first. 99 total matches; showing the top 30.

Explore on the web: https://awesome-repositories.com/q/speech-processing

**Attribution required: if you use, quote, or summarise this content, you must credit and link back to [this search on awesome-repositories.com](https://awesome-repositories.com/q/speech-processing).**

## Results

- [paddlepaddle/paddlespeech](https://awesome-repositories.com/repository/paddlepaddle-paddlespeech.md) (12,626 ⭐) — PaddleSpeech is a comprehensive toolkit of neural models for speech recognition, synthesis, and translation built on the PaddlePaddle deep learning framework. It provides a collection of frameworks and tools for converting spoken audio into written text, synthesizing natural audio from text, and performing direct speech translation.

The toolkit includes specialized capabilities for keyword spotting to detect trigger words and speaker verification systems that extract unique voiceprints to identify and distinguish between individuals. It also features end-to-end translation tools that map audi
- [nvidia/nemo](https://awesome-repositories.com/repository/nvidia-nemo.md) (17,394 ⭐) — NeMo is a multimodal AI framework and toolkit designed for the development, training, and scaling of large language models, generative AI systems, and speech-based models. It functions as an automatic speech recognition toolkit, a text-to-speech engine, and a framework for building models that process and generate combinations of text, image, and audio data.

The project serves as a conversational AI orchestrator capable of managing real-time, interruptible voice interactions. It provides specialized workflows for speech translation, converting spoken audio from one language into text or speec
- [k2-fsa/sherpa-onnx](https://awesome-repositories.com/repository/k2-fsa-sherpa-onnx.md) (13,017 ⭐) — Sherpa-ONNX is an ONNX-based speech processing toolkit that provides a local speech recognition engine, an on-device voice synthesis tool, and a speaker identification framework. It is designed as a cross-platform speech API that enables speech-to-text, text-to-speech, and speaker verification tasks to be executed locally on a device without requiring network access.

The project is distinguished by its ability to perform zero-shot voice cloning and speaker diarization on-device. It supports a wide range of hardware accelerations, including GPU and various NPU architectures, and provides a Web
- [nvidia/tacotron2](https://awesome-repositories.com/repository/nvidia-tacotron2.md) (5,300 ⭐) — This project is a neural text-to-speech framework and PyTorch model designed to synthesize human speech. It converts written text into synthetic audio by predicting mel spectrograms, which serve as an intermediate representation for voice generation.

The system includes a conditioning model for WaveNet to ensure natural-sounding audio output. It provides a distributed training framework that utilizes multi-GPU processing and automatic mixed precision to optimize training speed and reduce memory usage.

The project covers the full pipeline of neural speech synthesis, from model training using
- [corentinj/real-time-voice-cloning](https://awesome-repositories.com/repository/corentinj-real-time-voice-cloning.md) (59,918 ⭐) — This project is a neural text-to-speech engine and voice cloning toolkit designed to generate synthetic speech that mimics the vocal characteristics of a target speaker. It functions as a real-time audio synthesizer, utilizing a deep learning pipeline to convert written text into high-fidelity speech output with minimal latency.

The system employs a transfer learning framework that leverages pre-trained speaker verification models to adapt synthesis to new, unseen vocal identities. By using an encoder-based speaker embedding process, the toolkit maps variable-length audio samples into a laten
- [sesameailabs/csm](https://awesome-repositories.com/repository/sesameailabs-csm.md) (14,669 ⭐) — CSM is a conversational speech generation model and text-to-speech engine that converts text and audio inputs into synthetic speech. It utilizes a large language model architecture to predict and decode audio tokens for voice synthesis.

The system functions as a zero-shot voice cloner, replicating specific speaker identities using short audio samples without requiring additional training. This enables precise control over speaker identity and the creation of synthetic speech that mimics a specific person.

The model covers conversational speech synthesis and text-to-speech generation, transfo
- [speechbrain/speechbrain](https://awesome-repositories.com/repository/speechbrain-speechbrain.md) (11,624 ⭐) — SpeechBrain is an all-in-one deep learning toolkit designed for speech and audio processing. Built as a modular library, it provides a structured environment for developing, training, and deploying neural network models across a wide range of tasks, including automatic speech recognition, speaker identification, and audio enhancement.

The framework distinguishes itself through a configuration-driven approach that separates model architecture and training hyperparameters from application logic. By utilizing externalized configuration files and standardized recipes, it enables reproducible rese
- [funaudiollm/sensevoice](https://awesome-repositories.com/repository/funaudiollm-sensevoice.md) (7,536 ⭐) — SenseVoice is a multilingual speech large language model designed for audio transcription, speaker diarization, and emotion recognition. It functions as an automatic speech recognition system that converts spoken audio into text across multiple languages.

The system distinguishes itself by integrating acoustic event detection and speech emotion recognition, allowing it to identify non-speech sounds, such as laughter or applause, and discrete emotional states. It also includes a framework for speaker diarization to track and label different speakers within a single recording.

The project's ca
- [microsoft/vibevoice](https://awesome-repositories.com/repository/microsoft-vibevoice.md) (49,394 ⭐) — VibeVoice is a generative artificial intelligence platform designed for text-to-speech synthesis. It functions as a neural audio generation framework that converts written text into natural-sounding spoken audio, specifically engineered to maintain consistent vocal characteristics and narrative prosody across extended passages of content.

The system distinguishes itself through its ability to generate long-form conversational speech while preserving speaker identity and linguistic content. By utilizing latent space disentanglement, the model separates speaker traits from the input text, allow
- [espnet/espnet](https://awesome-repositories.com/repository/espnet-espnet.md) (9,861 ⭐) — ESPnet is a comprehensive speech processing toolkit and PyTorch-based trainer designed for building end-to-end speech recognition, synthesis, and translation models. It provides a structured framework for developing automatic speech recognition systems using transducer and encoder-decoder architectures, alongside engines for text-to-speech synthesis and speech translation pipelines.

The project distinguishes itself through a recipe-based workflow execution system that ensures experimental reproducibility by running standardized sequences of scripts for data preparation and model training. It
- [nvidia-nemo/nemo](https://awesome-repositories.com/repository/nvidia-nemo-nemo.md) (17,389 ⭐) — NeMo is a comprehensive framework designed for the development, training, and deployment of large-scale conversational and generative artificial intelligence models. It provides an integrated platform for building multimodal systems, encompassing speech processing, language modeling, and reinforcement learning alignment. The framework is built to handle the entire lifecycle of AI development, from data curation and model pretraining to production-ready service deployment.

The platform distinguishes itself through advanced distributed training capabilities, including tensor and pipeline parall
- [pannous/tensorflow-speech-recognition](https://awesome-repositories.com/repository/pannous-tensorflow-speech-recognition.md) (2,172 ⭐) — This library provides a deep learning framework for training neural networks to perform speech recognition and audio classification. It utilizes sequence-to-sequence architectures to map variable-length audio inputs into text or numerical outputs, enabling the development of custom speech-to-text transcription models.

The project distinguishes itself through integrated audio processing capabilities that transform raw waveforms into spectrograms and high-dimensional numerical vectors. These tools allow for the extraction of unique vocal characteristics to identify speakers, as well as the clas
- [netease-youdao/emotivoice](https://awesome-repositories.com/repository/netease-youdao-emotivoice.md) (8,446 ⭐) — EmotiVoice is an emotional text-to-speech engine and bilingual speech synthesizer designed to generate synthetic audio in English and Chinese. It utilizes a deep learning architecture to produce high-fidelity speech with controllable emotional states and timbres.

The project includes a voice cloning framework for replicating specific speaker identities by training custom acoustic models on personal audio datasets. It employs a jointly-trained acoustic-vocoder pipeline and style-embedding-based synthesis to manage expression and reduce audio artifacts.

The system covers a broad range of speec
- [coqui-ai/tts](https://awesome-repositories.com/repository/coqui-ai-tts.md) (45,568 ⭐) — This project is a deep learning text-to-speech toolkit used for training and deploying neural speech synthesis models. It provides a comprehensive framework for converting written text into spoken audio, utilizing neural vocoders to transform synthesized spectrograms into high-fidelity audio waveforms.

The toolkit includes a voice cloning system that replicates specific human voices by extracting speaker embeddings from short audio samples. It also supports multi-speaker audio synthesis, allowing the generation of speech across different vocal identities using specialized model architectures.
- [resemble-ai/chatterbox](https://awesome-repositories.com/repository/resemble-ai-chatterbox.md) (22,751 ⭐) — Chatterbox is a comprehensive machine learning platform designed for multilingual speech synthesis and real-time audio generation. It functions as an engine that converts text into natural-sounding speech, capable of replicating specific human vocal characteristics and emotional expressions from short audio samples.

The platform distinguishes itself through advanced control over the synthesis process, allowing for the manipulation of emotional intensity and the injection of non-verbal vocalizations such as laughter or coughing. It is engineered for low-latency performance, utilizing an optimi
- [neonbjb/tortoise-tts](https://awesome-repositories.com/repository/neonbjb-tortoise-tts.md) (14,864 ⭐) — Tortoise-tts is a neural text-to-speech engine and voice cloning toolkit designed for high-quality audio generation. It functions as a zero-shot synthesis system, meaning it can generate speech for unseen speakers without requiring additional training or fine-tuning for each new voice.

The system specializes in replicating human vocal characteristics using small sets of reference audio clips. It allows for the extraction of voice latents to mimic specific speakers, the generation of random synthetic identities, and the blending of multiple voice profiles to create hybrid vocal identities.

Th
- [openai/whisper](https://awesome-repositories.com/repository/openai-whisper.md) (102,828 ⭐) — This project is a speech recognition and translation engine that utilizes a sequence-to-sequence transformer architecture to convert audio into text. It is built upon a weakly supervised learning framework, which leverages large-scale, unlabelled audio-transcript data to create generalized speech representations capable of performing simultaneous transcription, language identification, and translation.

The system distinguishes itself through a unified multi-task modeling approach that shares token sequences across different objectives, allowing it to handle diverse languages and vocabularies
- [facebookresearch/wav2letter](https://awesome-repositories.com/repository/facebookresearch-wav2letter.md) (6,444 ⭐) — wav2letter is an automatic speech recognition toolkit and deep learning framework designed to convert audio speech signals into written text. It functions as a distributed training system and an inference engine for building and deploying neural network architectures.

The system enables the training of large-scale speech models across multiple compute nodes using custom architecture files and structured recipes. It includes an inference engine that allows these trained models to be executed within Python workflows to transform audio sequences into text.

The framework covers the full speech r
- [kaldi-asr/kaldi](https://awesome-repositories.com/repository/kaldi-asr-kaldi.md) (15,415 ⭐) — Kaldi is an automatic speech recognition toolkit used to train and deploy models that convert spoken audio into text. It functions as a framework for designing and evaluating acoustic and language models through a structured pipeline of processing tools.

The system acts as a cross-platform speech engine, capable of compiling recognition logic for Android and WebAssembly to enable execution on mobile devices and web browsers. It also includes a dedicated converter for migrating speech recognition models from the HTK format into a compatible internal structure.

The toolkit covers a broad range
- [nari-labs/dia](https://awesome-repositories.com/repository/nari-labs-dia.md) (19,324 ⭐) — Dia is a generative AI audio tool and text-to-speech synthesis engine designed for the production-ready deployment of machine learning models. It provides a framework for creating lifelike synthetic speech by conditioning generation on reference audio samples to replicate specific vocal characteristics, emotional tones, and delivery styles.

The system distinguishes itself through its ability to perform custom voice cloning and precise control over audio output. Users can adjust generation parameters such as temperature and guidance scale to modify the pacing, creativity, and style of the synt
- [fishaudio/fish-speech](https://awesome-repositories.com/repository/fishaudio-fish-speech.md) (24,928 ⭐) — This project is a generative speech synthesis engine that converts text into high-fidelity human speech. It utilizes a two-stage autoregressive transformer architecture that separates semantic token prediction from acoustic detail reconstruction to balance linguistic accuracy with audio quality. The system is designed to support multilingual output and conversational AI development, enabling the generation of context-aware speech that maintains flow across multiple dialogue turns.

The platform distinguishes itself through a production-ready inference server that employs continuous batching to
- [rvc-boss/gpt-sovits](https://awesome-repositories.com/repository/rvc-boss-gpt-sovits.md) (58,724 ⭐) — GPT-SoVITS is a text-to-speech synthesis engine and voice cloning toolkit designed for generating natural-sounding human speech. It functions as a neural audio processing pipeline that maps input text to high-fidelity audio waveforms, utilizing conditional variational autoencoders and flow-based decoders to ensure expressive output.

The platform distinguishes itself through its ability to perform few-shot voice cloning and cross-lingual speech generation, allowing users to maintain a specific speaker's vocal identity and emotional delivery across multiple languages. By employing cross-modal l
- [babysor/mockingbird](https://awesome-repositories.com/repository/babysor-mockingbird.md) (36,903 ⭐) — MockingBird is an AI voice cloning tool and text-to-speech system designed to generate synthetic speech. It functions as a voice synthesis trainer for building custom models from audio datasets, a command-line generator for producing audio files, and a text-to-speech server for remote application integration.

The project specializes in real-time voice cloning, which extracts vocal characteristics from short audio samples to mimic a target speaker's unique timbre. It utilizes reference-driven audio synthesis to condition pre-trained models on specific audio samples, allowing for the generation
- [jamiepine/voicebox](https://awesome-repositories.com/repository/jamiepine-voicebox.md) (30,041 ⭐) — Voicebox is a local speech processing system that provides text-to-speech generation, speech-to-text transcription, and voice cloning. It utilizes local machine learning inference and GPU acceleration to process audio and text data without relying on external API calls.

The project features a voice cloning toolkit for creating synthetic profiles from audio samples and a timeline-based voice editor for composing multi-character conversations. It also includes an AI voice management API that allows external applications and AI agents to programmatically manage voice profiles and generate speech
- [tensorspeech/tensorflowtts](https://awesome-repositories.com/repository/tensorspeech-tensorflowtts.md) (3,993 ⭐) — TensorFlowTTS is a neural speech synthesis framework used to convert text into high-fidelity audio waveforms. It provides a toolkit for training and fine-tuning sequence-to-sequence or generative adversarial network architectures to produce natural sounding speech.

The system includes neural vocoder implementations that transform intermediate acoustic representations into final audio waveforms. It also features playback speed control to adjust the rate of synthesized speech output.

The framework covers the end-to-end pipeline for speech synthesis, including audio data preprocessing to create
- [ggml-org/whisper.cpp](https://awesome-repositories.com/repository/ggml-org-whisper-cpp.md) (50,770 ⭐) — Whisper.cpp is a high-performance, local-first speech recognition engine designed to run large-scale machine learning models on consumer hardware. It functions as a portable library that converts audio into text, supporting both static file transcription and real-time stream processing. By utilizing a lightweight inference engine and weight quantization, the project minimizes memory and compute overhead, allowing for efficient execution without reliance on external cloud APIs or internet connectivity.

The project distinguishes itself through a hardware-agnostic compute abstraction that offloa
- [snakers4/silero-models](https://awesome-repositories.com/repository/snakers4-silero-models.md) (5,977 ⭐) — This is a collection of pre-trained neural models for speech recognition, synthesis, and voice activity detection. It provides a library of assets designed for speech-to-text, text-to-speech, and the identification of human speech segments within audio.

The project features text-to-speech synthesis with support for multiple languages and the use of Speech Synthesis Markup Language to control prosody, pitch, and timing. For speech recognition, the system includes capabilities for transcribing audio to text with word-level timestamp extraction and an automated punctuation restorer to insert cap
- [swivid/f5-tts](https://awesome-repositories.com/repository/swivid-f5-tts.md) (14,798 ⭐) — F5-TTS is a text-to-speech system that utilizes a flow matching engine and diffusion transformers to generate fluent synthetic speech. It functions as a multilingual speech synthesizer and neural training framework, providing tools for voice cloning and high-performance inference serving.

The project distinguishes itself through a voice cloning toolkit capable of mimicking specific speaker characteristics and tones from reference audio clips. It supports cross-lingual generation, allowing for the synthesis of audio across various global languages or the mixing of multiple languages within a s
- [voice-cloning-app/voice-cloning-app](https://awesome-repositories.com/repository/voice-cloning-app-voice-cloning-app.md) (1,438 ⭐) — This application is a platform for AI voice synthesis and neural voice cloning. It provides a comprehensive toolkit for converting text into natural-sounding human speech by applying custom-trained neural network models to specific audio samples.

The system facilitates the entire lifecycle of voice model development, including the preparation of raw audiobooks and video transcriptions into structured training datasets. It supports the training of these models on local or remote hardware, utilizing multi-GPU distributed processing to handle large-scale data and accelerate model convergence.

B
- [soniqo/speech-swift](https://awesome-repositories.com/repository/soniqo-speech-swift.md) (896 ⭐) — This project is a comprehensive toolkit for on-device speech recognition, synthesis, and audio processing, specifically engineered for Apple Silicon. It provides a framework for building real-time, full-duplex voice agents that operate entirely offline, leveraging native hardware acceleration to maintain performance and privacy. By utilizing optimized machine learning models, the library enables local execution of complex audio tasks without reliance on external cloud services.

The library distinguishes itself through its specialized focus on local, high-performance voice interaction. It incl