Voice activity detection

Silero VAD is a voice activity detection model and deep learning speech classifier designed to distinguish human speech from silence across diverse languages and noisy environments. It functions as a pre-trained neural network capable of identifying speech segments within both static audio recordings and real-time data streams. The project includes a language identification tool for classifying spoken languages and a framework for fine-tuning audio models. It provides utilities for optimizing detection thresholds using validation datasets and retraining the model with custom labeled audio to

Sherpa-ncnn is an edge-based speech recognition and synthesis engine designed to run neural network models locally on mobile, embedded, and desktop hardware. It provides a cross-platform framework for offline speech-to-text transcription and text-to-speech synthesis, ensuring that all audio processing occurs on-device without requiring an internet connection or external cloud services. The project distinguishes itself through its use of the ncnn inference engine, which is optimized for low-latency execution on resource-constrained devices. It incorporates on-device model quantization to reduc

This project is a comprehensive toolkit for on-device speech recognition, synthesis, and audio processing, specifically engineered for Apple Silicon. It provides a framework for building real-time, full-duplex voice agents that operate entirely offline, leveraging native hardware acceleration to maintain performance and privacy. By utilizing optimized machine learning models, the library enables local execution of complex audio tasks without reliance on external cloud services. The library distinguishes itself through its specialized focus on local, high-performance voice interaction. It incl

FastRTC is a Python framework for building low-latency, bidirectional audio and video streams. It serves as a real-time communication library that provides a wrapper for WebRTC media servers, allowing users to create streaming applications with integrated media handling. The project distinguishes itself by providing a gateway for telephony integration, which maps temporary phone numbers to streaming media endpoints. It also includes built-in voice activity detection to manage automatic turn-taking and speech boundary identification in real-time conversations. The library supports mounting me

SpeechBrain is an all-in-one deep learning toolkit designed for speech and audio processing. Built as a modular library, it provides a structured environment for developing, training, and deploying neural network models across a wide range of tasks, including automatic speech recognition, speaker identification, and audio enhancement. The framework distinguishes itself through a configuration-driven approach that separates model architecture and training hyperparameters from application logic. By utilizing externalized configuration files and standardized recipes, it enables reproducible rese

Pyannote.audio is a PyTorch toolkit for speaker diarization, speaker identification, and speech activity detection. Its primary purpose is to partition audio recordings into segments and assign each segment to a specific speaker identity to determine who spoke when. The project includes a framework for classifying speaker identities and a pipeline for distinguishing human speech from background noise. It provides specialized tools for handling symmetric-overlap speech, where multiple speakers talk simultaneously, and employs learnable band-pass filters for raw waveform feature extraction. Th

Porcupine is an on-device wake word detection engine that listens for a specific spoken phrase in real-time audio and triggers actions, all processed locally without any cloud connectivity. It includes a custom wake word model creator that generates production-ready models from just a few spoken examples in seconds, requiring no training data. Beyond wake word detection, Porcupine also provides on-device speech recognition for real-time transcription with custom vocabulary, an on-device audio content searcher that indexes and finds spoken phrases in audio files or streams, and a lightweight vo

This library is a browser-based utility for real-time voice activity detection. It monitors live microphone input to distinguish between background noise and human vocalizations, enabling applications to identify when a person begins or stops speaking. The project utilizes WebAssembly to execute signal processing logic within the browser sandbox, ensuring high-performance analysis of audio buffers. It integrates directly with the native audio graph to capture raw pulse-code modulation data, employing an energy-threshold approach to identify speech boundaries. The library provides an event-dr

Sherpa-ONNX is an ONNX-based speech processing toolkit that provides a local speech recognition engine, an on-device voice synthesis tool, and a speaker identification framework. It is designed as a cross-platform speech API that enables speech-to-text, text-to-speech, and speaker verification tasks to be executed locally on a device without requiring network access. The project is distinguished by its ability to perform zero-shot voice cloning and speaker diarization on-device. It supports a wide range of hardware accelerations, including GPU and various NPU architectures, and provides a Web

RealtimeSTT is a local speech-to-text engine and real-time automatic speech recognition server. It utilizes transformer-based recognition and omnilingual pipelines to convert live audio streams into text, providing a WebSocket-based streaming API for raw PCM audio transmission. The project is distinguished by a dual-backend transcription pipeline that uses a lightweight engine for immediate partial suggestions and a heavier model for final high-accuracy results. It includes a wake word detection system to trigger recording and employs a shared-resource inference model to distribute heavy spee

AudioKit is an audio framework for iOS, macOS, and tvOS that provides tools for digital audio synthesis, signal processing, and audio analysis. It functions as a synthesis engine for generating audio waveforms and textures, a processing library for modifying tonal characteristics, and a toolkit for extracting frequency and amplitude data from sonic signals. The framework utilizes a modular node architecture and graph-based signal routing to connect audio generators, processors, and outputs. It wraps low-level audio primitives in high-level classes to facilitate sound generation and modificati

NoiseTorch is a cross-platform audio processor and real-time noise filter designed to suppress ambient sound from audio streams. It functions as a virtual microphone noise suppressor and routing tool, capturing system audio sources and directing filtered signals into virtual input or output devices. The application uses a recurrent neural network to distinguish between human speech and ambient noise. It provides a virtual denoising microphone that removes background noise from a selected input, alongside tools for filtering audio output streams. The system includes capabilities for audio dev