KoljaB/RealtimeSTT

Features

• Real-Time Speech Processing - Provides a complete real-time pipeline for converting live audio streams into text using local models.
• Audio Transcription - Converts raw audio chunks from files or websockets into text by resampling audio to the required processing rate.
• Real-Time Transcription - Performs instantaneous conversion of live microphone audio streams into text transcripts via a persistent connection.
• Transcription APIs - Provides a WebSocket-based streaming server that offers programmatic transcription capabilities for integration into external applications.
• Transformer-Based ASR - Converts live audio to text using pre-trained transformer model families with specific generation settings.
• Local Speech-to-Text - Runs transcription inference on local hardware to ensure data privacy and remove external API dependencies.
• Automatic Speech Recognition - Utilizes transformer-based recognition and omnilingual pipelines for high-accuracy real-time transcription.
• Dual-Model Transcription Pipelines - Produces fast, preliminary transcriptions using a lightweight model while a larger model processes the final result.
• Incremental Transcription Previews - Generates incremental text results during active speech to show immediate progress before final transcription completes.
• Streaming Transcription Inference - Processes audio frames through a low-latency inference pipeline for immediate transcription previews.
• Hybrid Precision Pipelines - Utilizes a dual-backend pipeline with a lightweight engine for immediate partials and a heavy model for final accuracy.
• Speech Processing WebSocket Servers - Implements a WebSocket server that exposes real-time speech-to-text capabilities.
• Wake Word Detection - Monitors audio streams for specific activation phrases to trigger recording only after a wake word is spoken.
• Realtime Speech Translation - Provides real-time conversion of spoken audio from one language into text in another language.
• Real-Time Speech Translation - Converts spoken audio from one language into text in another language in real time.
• Speaker Diarization - Distinguishes between multiple voices in a single audio stream to attribute transcribed text to specific speakers.
• Speech-to-Text Integrations - Utilizes an omnilingual pipeline that supports various model sizes and compute types for multi-language transcription.
• Voice Activity Detection - Detects speech boundaries to automatically trigger recording and transcription cycles using sensitivity filters.
• Runtime Resource Sharing - Loads heavy speech models into memory once and shares them across multiple concurrent user sessions to minimize overhead.
• Multi-User Session Isolation - Ships a multi-user web server that isolates sessions while sharing inference resources for remote transcription.
• PCM Audio Streaming - Accepts raw PCM audio over WebSockets and returns incremental transcription results.
• Audio Transcription WebSockets - Implements a WebSocket connection to transmit live PCM audio for real-time transcription results.
• Inference Stream Multiplexing - Manages multiple concurrent user sessions by isolating audio buffers while sharing model weight memory for inference.
• Inference Session Isolation - Isolates audio buffers and transcription states for multiple simultaneous users while sharing a single inference engine.
• Transcription Engine Adapters - Implements modular interfaces that allow swapping different speech-to-text engines without modifying core application logic.
• Split-Pipeline Transcription - Allows using separate transcription engines for partial and final results to balance processing speed with accuracy.

Star history