modelscope/FunASR
FunASR
FunASR is an automatic speech recognition toolkit and multilingual speech-to-text engine designed to convert spoken audio into written text across more than fifty languages. It provides a framework for speaker diarization, an OpenAI-compatible transcription API for local server hosting, and speech models compatible with the ONNX format.
The project distinguishes itself by supporting high-performance inference on edge hardware via self-contained binaries and portable model exports. It incorporates specialized capabilities for natural speech generation with adjustable timbre and emotional expression, as well as the ability to capture live microphone audio for direct voice-to-text input automation.
The toolkit covers a broad range of audio analysis and processing capabilities, including voice activity detection, audio event and emotion detection, and punctuation restoration. It also includes tools for automated video captioning through the generation of timed subtitle files and distributed model fine-tuning to improve recognition accuracy using custom datasets.
Features
- Automatic Speech Recognition - Provides a comprehensive toolkit for automatic speech recognition, including local server hosting and multilingual support.
- Multilingual Transcription - Provides high-performance transcription services supporting over fifty languages with automatic language detection.
- Real-Time Transcription - Processes live audio streams to provide immediate, low-latency text transcriptions.
- Multilingual Speech-to-Text - Implements an engine capable of detecting languages and transcribing speech across fifty different languages.
- Multilingual Models - Employs a shared representation space to recognize and transcribe speech across more than fifty languages.
- Speaker Diarization - Identifies and labels different speakers within an audio stream to track participants over time.
- Speech Transcription - Utilizes advanced model architectures to transcribe spoken audio into text across multiple languages.
- Streaming Transcription Inference - Enables low-latency text output during live speech by processing audio in small incremental segments.
- Voice Activity Detection - Provides precise identification of the start and end timestamps of speech segments in audio streams.
- OpenAI-Compatible APIs - Exposes transcription endpoints using a standard OpenAI-compatible API format for easy integration with external tools.
- Transcription APIs - Provides a local server endpoint for programmatic access to automated speech transcription services.
- Audio Segmenting - Splits audio files into distinct sections by detecting voice activity and adjusting for silence thresholds.
- Controllable Speech Generation - Produces synthetic voice audio with adjustable controls for timbre and emotional expression.
- Inference Acceleration - Increases transcription throughput and reduces latency using parallel batch decoding techniques.
- Spoken Language Detection - Identifies the specific language being spoken from a wide range of supported multilingual options.
- Edge Model - Executes speech models on CPU or edge devices using self-contained binaries for high-performance inference.
- Speech Model Fine-Tuning - Supports distributed fine-tuning of speech models using custom datasets to increase recognition accuracy.
- Model Export and Portability - Supports exporting optimized speech models to ONNX format for high-performance inference on edge hardware.
- ONNX Model Exporters - Converts trained speech model weights into the portable ONNX format for execution across diverse hardware.
- ONNX Model Exports - Provides speech models exported to the Open Neural Network Exchange format for cross-hardware compatibility.
- Model Export Formats - Converts trained models into universal industry formats for deployment via containerized runtimes.
- Voice-to-Text Input Automation - Captures live microphone audio and inserts recognized words directly into the active cursor position of an application.
- Timestamped Subtitle Generators - Generates timed subtitle files in SRT or VTT formats by aligning transcribed text with audio timestamps.
- Inference Batching - Increases hardware throughput and inference speed by processing multiple audio streams simultaneously in batches.
- Audio Event Detection - Identifies non-speech sounds and speaker emotions to provide comprehensive contextual audio information.
- AI and Agents - Industrial-grade speech recognition toolkit with 170x realtime speed.
- Language Model Development - Production-grade toolkit for automatic speech recognition.
- Media and Content Creation - Performs industrial-grade speech recognition and transcription.
