snakers4/silero-models
Silero Models
This is a collection of pre-trained neural models for speech recognition, synthesis, and voice activity detection. It provides a library of assets designed for speech-to-text, text-to-speech, and the identification of human speech segments within audio.
The project features text-to-speech synthesis with support for multiple languages and the use of Speech Synthesis Markup Language to control prosody, pitch, and timing. For speech recognition, the system includes capabilities for transcribing audio to text with word-level timestamp extraction and an automated punctuation restorer to insert capitalization and punctuation into raw text.
The models are exported to the Open Neural Network Exchange format and TorchScript to enable high-performance execution across different hardware accelerators and operating systems.
Features
- Automatic Speech Recognition - Provides pre-trained neural models for converting spoken audio into written text.
- Text-to-Speech Synthesis - Provides pre-trained neural models that convert written text into natural sounding spoken audio.
- ONNX Model Exporters - Exports neural models to the standardized ONNX format for high-performance execution across various hardware.
- ONNX Model Exports - Delivers speech models exported to ONNX format for cross-engine compatibility and performance.
- ONNX Runtime Inference - Enables the execution of audio models across different operating systems using the ONNX runtime.
- TorchScript Exports - Packages models using TorchScript to enable high-performance inference without requiring a Python runtime.
- Pre-trained Speech Models - Provides a comprehensive library of pre-trained neural models for speech recognition, synthesis, and activity detection.
- Punctuation Restoration - Includes a pre-trained model that automatically inserts capitalization and punctuation into raw speech transcripts.
- Multilingual Synthesis - Provides synthesis models capable of generating speech across a wide variety of regional and minority languages.
- Speech to Text Transcription - Converts spoken audio into text using models optimized for performance and size.
- Voice Activity Detection - Includes a model for detecting human speech boundaries to isolate voice from background noise.
- Word-Level Timestamps - Provides precise start and end times for individual words during the audio transcription process.
- Prosody Control - Allows adjustment of pitch, rate, and timing of synthesized speech to achieve natural inflection.
- Speech Prosody Generation - Automatically applies word stress and resolves homographs to improve the natural pronunciation of spoken language.
- SSML Conversions - Supports Speech Synthesis Markup Language (SSML) to provide precise control over pitch, timing, and prosody.
- Phoneme-Based Speech Processors - Implements a synthesis pipeline that converts text into phonetic representations to ensure natural pronunciation.
- Speech Synthesis Markup Controls - Integrates markup controls to refine the prosody and pronunciation of synthesized speech.
