k2-fsa/sherpa-onnx

Features

• Local Inference - Executes all speech processing tasks directly on the local device without requiring network access.
• Speech Recognition Systems - Provides an offline inference system that converts spoken audio to text across multiple platforms.
• Voice Activity Detection - Identifies speech segments in audio files or streams to optimize transcription and filter silence.
• Real-Time Transcription - Processes live microphone input for low-latency transcription, keyword spotting, and noise reduction.
• Local Speech-to-Text - Converts spoken audio into text locally using ONNX models without an internet connection.
• ONNX Model Exporters - Transforms speech models into the standardized ONNX format for cross-platform local execution.
• ONNX Runtime Inference - Uses the cross-platform ONNX runtime to execute speech models without requiring full ML installations.
• Local Execution - Provides offline speech-to-text conversion utilizing local hardware for privacy and reduced latency.
• Speech Recognition APIs - Implements speech recognition and processing capabilities specifically for applications developed in Object Pascal.
• Voice Synthesis - Generates synthetic speech from text and clones voices using on-device local models.
• On-Device Inference Engines - Provides a runtime for classifying sound events and detecting languages on edge hardware.
• Speaker Diarization - Segments audio recordings by speaker identity to determine who spoke and when.
• Speaker Identification Frameworks - Provides a framework for extracting voice embeddings to perform speaker verification and identity matching.
• Speech Processing APIs - Provides a comprehensive Java interface for speech-to-text, text-to-speech, and speaker identification tasks.
• Speech Processing Toolkits - Offers a complete toolkit for local speech-to-text, text-to-speech, and speaker identification via ONNX.
• Speech Recognition APIs - Provides a Kotlin interface for running speech processing models to translate audio into text.
• Speech-to-Text Integrations - Provides a Java interface for transcribing audio into text across various operating systems and hardware.
• Incremental Processing - Converts audio incrementally to produce partial text results for real-time transcription.
• Text-to-Speech - Generates synthetic spoken audio from text locally using pretrained voice models.
• Local Speech Synthesis - Generates spoken audio from text locally on mobile devices using on-device models.
• Wake Word Detection - Monitors audio streams for specific keywords to trigger actions in voice assistants.
• Custom Phrase Detection - Identifies user-defined words in an audio stream using a beam search decoder.
• Voice Profile Managers - Extracts and stores fixed-dimensional vectors from audio utterances to create voice profiles.
• Audio Noise Cancellation - Implements real-time and offline techniques to remove background noise from speech recordings.
• Source Separation Tools - Provides tools to isolate individual source components, such as vocals, from mixed audio signals.
• Audio Transcription - Processes recorded wave files to generate written transcripts of spoken content.
• Self-Hosted Server Integration - Implements a self-hosted WebSocket server that processes audio files into text using parallel batching.
• Audio Trigger Detection - Monitors audio streams for the occurrence of predefined words or phrases to trigger actions.
• Zero-Shot Voice Cloning - Synthesizes speech from text using a small sample of a target voice without additional training.
• Keyword Spotting - Detects specific predefined words or phrases within an audio signal.
• Spoken Language Detection - Detects the language being spoken in an audio recording using pre-trained models.
• Speech-to-Text Translation - Converts spoken audio in one language into written text in a different target language.
• Multilingual Speech-to-Text - Supports transcribing audio containing multiple languages and regional dialects into text.
• Speaker Embeddings - Converts audio waveforms into mathematical embedding vectors representing unique vocal characteristics.
• Embedding Management - Provides a system for registering, verifying, and removing speaker identities by managing voice embeddings.
• Embedding Similarity Analysis - Calculates cosine similarity scores between speaker embeddings to determine if voice samples match.
• Speaker Verification - Compares a voice embedding against stored profiles to verify the identity of the speaker.
• Speech Recognition Services - Ships a WebSocket server that provides real-time streaming speech-to-text decoding for remote clients.
• Multilingual Synthesis - Converts mixed language text into audio using a multi-speaker model.
• Voice Cloning - Generates synthetic speech that mimics the vocal characteristics of a target speaker using a reference clip.
• Cross-Platform Deployment Targets - Supports installation and execution across various operating systems with CPU and GPU acceleration.
• Speech Processing WebSocket Servers - Hosts WebSocket servers to provide remote streaming and batch transcription services to clients.
• WebSocket Servers - Hosts a WebSocket server to provide remote streaming and batch transcription services.
• WebAssembly Compilation - Compiles speech processing capabilities into WebAssembly modules for browser-based execution.
• WebAssembly Frameworks - Enables real-time speech-to-text functionality directly within web browsers using WebAssembly.
• Machine Learning - Speech and audio processing using next-gen Kaldi.

Star history