snakers4/silero-vad
Silero Vad
Silero VAD is a voice activity detection model and deep learning speech classifier designed to distinguish human speech from silence across diverse languages and noisy environments. It functions as a pre-trained neural network capable of identifying speech segments within both static audio recordings and real-time data streams.
The project includes a language identification tool for classifying spoken languages and a framework for fine-tuning audio models. It provides utilities for optimizing detection thresholds using validation datasets and retraining the model with custom labeled audio to improve accuracy.
The system covers audio analysis capabilities such as speech probability estimation, temporal timestamp identification, and audio segment extraction. It also handles automated preprocessing by isolating and merging speech chunks to remove silence.
Features
- Pre-trained Speech Models - Ships a pre-trained deep learning model designed to classify audio frames as speech or silence.
- Voice Activity Detection - Implements high-performance voice activity detection to identify speech boundaries in real-time and static audio streams.
- Speech Boundary Detection - Provides the ability to locate exact start and end timestamps of spoken segments within audio recordings.
- Deep Learning Classifiers - Uses a deep learning classifier to distinguish between human speech and silence across diverse environments.
- Spoken Language Detection - Identifies the specific language being spoken within an audio stream using pattern recognition.
- Detection Model Fine-Tuning - Improves speech detection accuracy by retraining models using specific audio datasets and labeled timestamps.
- Speech Model Fine-Tuning - Improves speech detection quality by retraining models using custom audio paths and time-stamped labels.
- Speech Detection Fine-Tuning Frameworks - Provides a framework for retraining voice detection models using custom labeled datasets and optimized thresholds.
- Real-Time Speech Processing - Implements a real-time processing pipeline for detecting speech activity within live audio streams.
- Speech Activity Detection - Identifies active voice segments within continuous audio streams using configurable confidence thresholds.
- Lossless Audio Streaming - Analyzes audio input incrementally to detect the presence of speech as it occurs in real-time.
- Speech Segment Extraction - Extracts speech timestamps and isolates voice segments from raw audio files.
- Speech-Based Silence Removal - Isolates and merges speech segments from a recording to remove silence before transcription.
- Audio Detection Thresholds - Provides utilities to calculate optimal speech trigger levels using labeled validation datasets.
- Prediction Thresholds - Calculates confidence scores for audio segments to determine if they exceed defined speech triggers.
- Decision Threshold Calibration - Calculates ideal input and output probability thresholds based on validation datasets to maximize detection accuracy.
- Speech Probability Scoring - Calculates a score between zero and one for each audio chunk to estimate the likelihood of human speech.
- Speech Segment Extraction - Isolates and merges detected speech chunks from an audio recording into a single continuous file.
- Audio and Speech Models - Locates portions of audio containing speech and handles resampling to ensure consistent input quality.
- Speech Boundary Timestamps - Locates the start and end times of speech segments within an audio file.
- Speech Probability Scoring - Calculates a numerical score for each audio window to estimate the likelihood of human speech.
- Audio Segment Offsets - Maps model output indices to temporal offsets to isolate specific voice segments from recordings.
- Voice activity detection - Listed in the “Voice activity detection” section of the Ailia Models awesome list.
