myshell-ai/MeloTTS
MeloTTS
MeloTTS is an open-source text-to-speech library that generates natural-sounding speech across six languages, with the ability to mix two languages within a single utterance. Its architecture combines a token-based text frontend with a language-agnostic acoustic model, enabling it to handle bilingual code-switching and produce streaming audio output in real time.
The system is designed to run efficiently on standard CPU hardware without requiring a dedicated GPU, using a lightweight neural network for real-time inference. It supports English, Spanish, French, Chinese, Japanese, and Korean, and can process mixed-language input such as Chinese and English within the same sentence by switching between language-specific acoustic models.
The library provides a freely available toolkit for developers to integrate speech synthesis into applications, with phoneme mapping that preserves language identity and prosodic boundaries across all supported languages.
Features
- TTS Engine Optimizations - A speech synthesis engine designed to run real-time inference on standard CPU hardware without requiring a dedicated GPU.
- Bilingual Speech Synthesizers - Handles text containing both Chinese and English within a single utterance for bilingual speech generation.
- CPU Inference Runtimes - Runs a lightweight neural network model designed for real-time speech synthesis on standard CPU hardware.
- Speech Inference Runtimes - Runs real-time text-to-speech synthesis on standard CPU hardware without requiring a dedicated GPU.
- Shared Acoustic Backbones - Uses a shared neural backbone trained on multilingual data to generate speech features independent of input language.
- Prosody-Preserving Tokenizers - Converts raw text into a tokenised representation that preserves language identity and prosodic boundaries for synthesis.
- CPU-Based Synthesizers - Runs real-time text-to-speech inference on standard CPU hardware without needing a dedicated GPU.
- Multilingual Text-to-Speech Engines - Converts written text into natural-sounding speech across multiple languages including English, Spanish, French, Chinese, Japanese, and Korean.
- Bilingual Code-Switching - Detects and processes mixed-language text within a single utterance by switching between language-specific acoustic models.
- Open Source Toolkits - A freely available toolkit for developers to integrate high-quality speech synthesis into applications with multi-language support.
- Grapheme-to-Phoneme Pipelines - Maps input text to language-specific phoneme sequences using a unified phonetic representation across six languages.
- Streaming Audio Generators - Produces audio output in small chunks during inference to minimise latency and enable real-time playback.
- AI & Machine Learning - Multi-lingual text-to-speech library.
