Awesome GitHub RepositoriesNeural Text-to-Speech Engines

This project is a neural text-to-speech engine and voice cloning toolkit designed to generate synthetic speech that mimics the vocal characteristics of a target speaker. It functions as a real-time audio synthesizer, utilizing a deep learning pipeline to convert written text into high-fidelity speech output with minimal latency. The system employs a transfer learning framework that leverages pre-trained speaker verification models to adapt synthesis to new, unseen vocal identities. By using an encoder-based speaker embedding process, the toolkit maps variable-length audio samples into a laten

Este proyecto es un kit de herramientas de aprendizaje profundo de texto a voz utilizado para entrenar y desplegar modelos de síntesis de voz neuronal. Proporciona un marco integral para convertir texto escrito en audio hablado, utilizando vocoders neuronales para transformar espectrogramas sintetizados en formas de onda de audio de alta fidelidad. El kit de herramientas incluye un sistema de clonación de voz que replica voces humanas específicas extrayendo incrustaciones de hablante de muestras de audio cortas. También admite la síntesis de audio de múltiples hablantes, lo que permite la generación de voz a través de diferentes identidades vocales utilizando arquitecturas de modelos especializadas. El sistema cubre todo el proceso de síntesis de voz, incluyendo herramientas para la curación de conjuntos de datos de voz, entrenamiento de modelos personalizados con seguimiento de rendimiento e interfaz de línea de comandos para la generación de audio. Para el acceso a la red, proporciona un servidor HTTP autohospedado para desplegar modelos de síntesis de voz como una API.

MockingBird is an AI voice cloning tool and text-to-speech system designed to generate synthetic speech. It functions as a voice synthesis trainer for building custom models from audio datasets, a command-line generator for producing audio files, and a text-to-speech server for remote application integration. The project specializes in real-time voice cloning, which extracts vocal characteristics from short audio samples to mimic a target speaker's unique timbre. It utilizes reference-driven audio synthesis to condition pre-trained models on specific audio samples, allowing for the generation

OpenVoice is a multilingual text-to-speech framework and voice cloning AI model designed for high-fidelity voice replication and low-latency audio generation. It functions as an instant speech synthesis engine that converts text to audio while replicating a specific speaker's tone and color. The system is distinguished by its ability to perform cross-lingual cloning, allowing the vocal characteristics of a reference speaker to be applied to speech in different languages regardless of the original training data. It utilizes a decoupled representation to separate the physical identity of a voic

This project is a singing voice conversion tool based on VITS generative modeling. It transforms the identity of a singing voice to a target speaker while preserving the original melody, lyrics, and intonation. The system distinguishes itself through hybrid voice synthesis, allowing for the blending of multiple speaker identities via linear model interpolation. It utilizes cluster-based feature retrieval to increase target voice similarity and employs a diffusion probabilistic model as a post-processor to remove electronic artifacts and improve vocal clarity. The software covers a broad rang

CosyVoice is a speech synthesis framework that utilizes large language models to generate expressive, multilingual audio. The system functions as an audio generation engine capable of producing natural-sounding speech across multiple languages while preserving regional dialects and specific emotional tones. The platform distinguishes itself through its zero-shot voice cloning capabilities, which allow for the creation of synthetic voice profiles from short audio samples without requiring additional model training. It provides fine-grained control over vocal attributes, enabling users to adjus

Dia is a generative AI audio tool and text-to-speech synthesis engine designed for the production-ready deployment of machine learning models. It provides a framework for creating lifelike synthetic speech by conditioning generation on reference audio samples to replicate specific vocal characteristics, emotional tones, and delivery styles. The system distinguishes itself through its ability to perform custom voice cloning and precise control over audio output. Users can adjust generation parameters such as temperature and guidance scale to modify the pacing, creativity, and style of the synt

Index-tts is a neural audio generation engine designed to convert written text into high-fidelity human speech. By utilizing deep learning models and phoneme-based sequence modeling, the system transforms text into natural-sounding audio waveforms suitable for a variety of accessibility and media applications. The platform functions as a server-side inference pipeline that provides a programmatic interface for integrating voice generation into external applications. It distinguishes itself through asynchronous audio streaming, which buffers and delivers generated speech chunks in real time to

Piper is a local neural text-to-speech engine designed to convert written text into natural human speech entirely on your own hardware. By utilizing a neural synthesis framework, it operates without the need for internet connectivity, ensuring that all audio generation remains private and secure. The system distinguishes itself through a modular architecture that allows for the dynamic loading of speaker embeddings and voice configurations. This enables users to switch between various vocal personas and styles without requiring a full reload of the core synthesis model. By processing input th

This project is a comprehensive suite for neural speech synthesis, featuring a deep learning text-to-speech engine, a neural speech synthesis trainer, and a voice cloning toolkit. It provides a system for synthesizing human-like speech from text using neural network models and high-fidelity vocoders. The suite includes a speech model conversion utility to transform deep learning models between different formats for deployment across various hardware runtimes. It also provides a self-contained HTTP server to expose pre-trained text-to-speech models as a remote audio API. Capabilities include

KittenTTS is a neural text-to-speech engine and text-to-audio synthesis tool that converts written text into spoken audio using lightweight neural network models. It functions as both a speech synthesizer and an audio file generator, producing spoken audio for offline playback. The system includes a text normalization processor that expands numbers and abbreviations into full spoken words to improve the naturalness of the synthesized speech. It supports diverse voice options and provides the ability to adjust playback speed.

ESPnet is a comprehensive speech processing toolkit and PyTorch-based trainer designed for building end-to-end speech recognition, synthesis, and translation models. It provides a structured framework for developing automatic speech recognition systems using transducer and encoder-decoder architectures, alongside engines for text-to-speech synthesis and speech translation pipelines. The project distinguishes itself through a recipe-based workflow execution system that ensures experimental reproducibility by running standardized sequences of scripts for data preparation and model training. It

This project is a GPU-accelerated speech engine and AI voice cloning tool. It functions as a text-to-speech synthesizer and voice-to-voice converter that replicates specific human voices to generate synthetic speech. The system creates digital voice profiles by analyzing short audio samples or capturing live microphone input. These profiles enable the transformation of existing audio recordings into a target speaker's voice or the synthesis of new audio from written text. The engine supports subtitle-based speech generation for batch processing and automated dubbing workflows. A web-based au

Higgs-audio is a generative text-to-speech engine that transforms text into natural conversational speech using large language model architectures. It functions as a multilingual speech synthesizer capable of generating high-fidelity audio across different languages with control over emotional tone and prosody. The system includes a voice cloning tool that creates synthetic replicas of specific speakers from short audio samples without requiring extensive model training. It also provides a streaming audio API designed to deliver generated speech incrementally to minimize playback delay. The

Neutts is a neural text-to-speech engine designed for real-time streaming output on edge devices such as phones and laptops. It supports voice cloning from short audio references, enabling zero-shot reproduction of a target speaker's voice, and can be fine-tuned or retrained from scratch for custom voices and styles. The system distinguishes itself through a decoder-only architecture that halves memory and accelerates generation on constrained hardware, combined with quantized model inference for reduced memory footprint. Its streaming decoder loop interleaves synthesis with playback, deliver

Kokoro is a lightweight neural text-to-speech engine that converts written text into spoken audio using a compact model designed for fast inference. It supports multiple languages through language-specific grapheme-to-phoneme conversion pipelines, and offers voice profile selection to change the character of the generated speech. The engine provides GPU acceleration on Apple Silicon hardware by setting a single environment variable, enabling faster inference on Mac M-series machines. It also includes pattern-based text segmentation, allowing input text to be split at user-defined delimiters t

Este proyecto es una implementación en TensorFlow de una red neuronal para la generación de formas de onda de audio crudas. Funciona como un modelo de síntesis de voz condicionado que produce muestras de audio sintéticas utilizando una arquitectura de red neuronal convolucional dilatada. El sistema admite el modelado de voz personalizado mediante la incorporación de condicionamiento global e identificadores categóricos durante el entrenamiento y la generación. Esto permite que el modelo imite hablantes específicos o características de audio distintas para aplicaciones de texto a voz neuronal. El framework cubre la síntesis de audio mediante aprendizaje profundo, incluyendo el procesamiento de datasets de audio, entrenamiento de modelos a partir de archivos de forma de onda y la generación de archivos de audio reproducibles. Utiliza componentes técnicos como convoluciones causales dilatadas, companding mu-law y salidas softmax cuantizadas para manejar dependencias de largo alcance en datos de audio.

VITS-fast-fine-tuning es un pipeline para adaptar modelos de síntesis de voz a voces objetivo específicas utilizando pequeños conjuntos de datos de audio. Funciona como una herramienta de adaptación rápida de hablante y un sintetizador de voz multilingüe capaz de generar audio hablado en diferentes idiomas. El sistema proporciona un framework para la conversión de voz muchos-a-muchos, transformando la identidad de un hablante en otro mientras se preserva el contenido lingüístico original. Permite la adaptación de una voz para texto-a-voz mediante el fine-tuning de un modelo pre-entrenado con clips de audio o fuentes de video. El proyecto cubre la síntesis de voz end-to-end y el procesamiento de audio, utilizando generación de formas de onda adversarias y búsqueda de alineación monótona para producir audio de alta fidelidad. Incorpora un predictor de duración estocástico para gestionar variaciones en el ritmo del habla y admite la transferencia de modelos pre-entrenados.

This project is a neural text-to-speech system and voice trainer that converts written text into spoken audio across a variety of global languages and regional dialects. It functions as an ONNX-based engine capable of performing fast offline inference and uses a phoneme-based controller to manage precise pronunciation. The system distinguishes itself through a comprehensive toolkit for neural voice training, allowing for the creation of custom single-speaker or multi-speaker models. It supports the export of these models to a standardized open format and provides hardware acceleration via gra