jamiepine/voicebox
Voicebox
Voicebox is a local speech processing system that provides text-to-speech generation, speech-to-text transcription, and voice cloning. It utilizes local machine learning inference and GPU acceleration to process audio and text data without relying on external API calls.
The project features a voice cloning toolkit for creating synthetic profiles from audio samples and a timeline-based voice editor for composing multi-character conversations. It also includes an AI voice management API that allows external applications and AI agents to programmatically manage voice profiles and generate speech.
Capabilities cover audio processing pipelines for effects like pitch shifts and reverb, as well as real-time and file-based transcription with filler word removal. The system supports persona-based dialogue generation, batch synthesis with prompt caching, and global text dictation for inserting transcripts directly into the operating system clipboard.
The processing engine can be hosted on local hardware or remote GPU servers.
Features
- Local Inference Engines - Executes machine learning models on local GPU hardware to ensure low-latency speech generation and privacy.
- Text-to-Speech - Converts text into natural sounding audio using local GPU acceleration to avoid cloud latency and fees.
- Voice Profiling - Enables the connection of AI agents to a local speech interface using specific voice profiles.
- Audio Transcription - Converts speech from audio files to text using local models with adjustable accuracy settings.
- Real-Time Transcription - Converts live microphone or system audio into text transcripts with real-time visualization.
- Local Speech-to-Text - Runs speech-to-text models on the host machine to convert audio streams into text offline.
- Local AI Inference - Executes machine learning models on personal GPU hardware to process audio and text without external API calls.
- Synthetic Speech Generation - Converts text to audio using cloned voices with support for high-volume batch processing.
- Local Speech Synthesis - Converts text into audio using local engines with granular controls for emotion and long-form content.
- Voice Cloning - Creates synthetic voice profiles that replicate specific human vocal characteristics from short audio samples.
- Voice Cloning Toolkits - Ships a comprehensive set of utilities for creating synthetic voice profiles from audio samples.
- Voice-Enabled Agents - Provides the ability to link AI agent profiles to specific cloned voice models for natural speech output.
- Voice Agents - Connects AI agents to local speech synthesis endpoints to provide them with distinct audible personas.
- Persona Identity Definitions - Assigns personality traits and descriptions to voice profiles to influence dialogue composition.
- Voice Profile Management - Organizes cloned voices with language tags and descriptions while supporting sample import and export.
- Programmatic APIs - Provides a network interface for programmatic voice profile management and speech generation for external applications.
- Programmatic Interfaces - Provides a REST API to programmatically manage voice profiles and generate speech without external fees.
- Voice Dictation - Inserts spoken transcripts into focused text fields or the clipboard using a global system shortcut.
- Narrative Timeline Editors - Arranges multiple voice tracks on a timeline to build conversations with inline trimming and splitting.
- Audio Narrative Compositors - Builds complex conversations and audio stories using a timeline editor with multiple cloned character voices.
- Audio Processing Pipelines - Routes synthetic speech through a sequential processing chain of pitch shifts and reverb effects.
- Multi-Track Audio Sequencers - Organizes multiple voice tracks and clips on a temporal axis for mixing multi-character conversations.
- Persona-Driven Dialogue Rewriting - Rewrites existing text or composes new lines based on a personality description assigned to a voice.
- Voice Arrangement Timelines - Provides a workspace for composing and mixing conversations between synthetic characters with trimming and effects.
- TTS API Endpoints - Exposes text-to-speech synthesis and voice profile management as network-accessible API endpoints.
Star history
How this analysis was created: This summary and feature list are AI-generated from collected project material and can contain mistakes. Stars, license and language are imported from GitHub. Inclusion does not mean that we have tested or audited this project. Check the source documentation for any feature you depend on. Learn more on our About page.
