cjpais/Handy
Handy
Handy is a local speech-to-text automation tool designed to convert spoken audio into text and inject it directly into active desktop applications. By running machine learning models entirely on the host hardware, it provides a private, offline-first environment for dictation and command execution. The system functions as a background service that manages microphone input, transcription state, and text output, enabling hands-free typing across various software environments.
The project distinguishes itself through a modular pipeline that integrates local language models for post-transcription refinement. Users can configure custom prompts to automatically format, translate, or correct raw speech output before it is inserted into the target application. This workflow is further enhanced by event-driven automation hooks, which allow the system to trigger custom scripts, keyboard shortcuts, or command sequences in response to transcription events.
Beyond core dictation, the software offers extensive control over the transcription environment, including hardware-aware audio management and real-time translation capabilities. It supports fine-grained adjustments to transcription accuracy, such as vocabulary correction for technical terminology and configurable input latency. The system also maintains a history of past sessions and provides tools for managing clipboard states and system memory usage.
Features
- Transcription Model Selectors - Provides configurable selection of local speech-to-text models to balance transcription accuracy and hardware performance.
- Local AI Inference - Executes speech-to-text models locally on host hardware to maintain data privacy and offline functionality.
- Speech-to-Text Engines - Converts spoken audio into written text locally and injects it into active applications.
- Transcription Refinement Pipelines - The application applies automated AI-driven corrections, formatting, or translation to raw speech-to-text output before pasting it into active applications for polished results.
- AI Writing Assistants - Processes transcribed speech through local language models to format, translate, or refine text.
- Audio Transcription - Converts spoken audio into text locally using selectable machine learning models.
- Keyboard Input Automation - Automates the insertion of transcribed text into active windows and legacy software.
- Desktop Automation - Triggers custom scripts and keyboard shortcuts to streamline text entry and command execution across desktop environments.
- Local Model Integrations - Connects to local language model services to ensure private, offline-first text processing.
- Multilingual Speech Translation - Converts foreign speech into target language text in real-time during the transcription process.
- Vocabulary Extension Sets - Allows users to define custom vocabulary lists to improve transcription accuracy for technical terminology and jargon.
- Transcription Input Orchestrators - Manages microphone input, transcription state, and clipboard history to enable hands-free text entry.
- System Input Handling - Injects text and simulates keyboard shortcuts into active applications via low-level OS APIs.
- Cross-Application Input Simulators - Simulates keyboard shortcuts and clipboard operations to reliably insert text into active applications.
- AI Text Refinement Pipelines - Refines and transforms raw transcribed text using local language models to improve formatting and grammar.
- Hardware Abstraction Layers - Provides a consistent interface for managing microphone selection and audio hardware profiles.
- Audio Hardware Profiles - Configures microphone inputs and audio behaviors to maintain consistent transcription quality.
- Global Hotkey Managers - Registers global hotkeys to trigger text transformation and editing independently of standard dictation.
- Transcription Event Hooks - Triggers custom scripts and keyboard shortcuts in response to transcription events.
- Desktop Applications - Provides CLI-based control for managing transcription states and desktop environment integration.
- Event-Driven Hooks - Triggers custom scripts and workflows in response to transcription events or hotkey signals.
- Recording Controls - Assigns custom keyboard triggers to interrupt or cancel active recording sessions.
- Transcription Correction Sensitivity - Adjusts the sensitivity of custom word corrections to balance between strict matching and aggressive replacement.
- Modular Pipeline Orchestration - Orchestrates a modular pipeline that chains transcription with secondary text transformation steps.
- Clipboard Management - Manages clipboard state by optionally restoring previous content after text insertion.
- Command Automation - Automates command submission by simulating keyboard shortcuts after text insertion.
- Task Automation Scripts - Pipes transcribed text directly into shell scripts for programmatic input processing.
- AI System Instructions - Defines custom system instructions to guide how language models process and format transcribed text.
- Audio Modality Controls - Controls audio input modalities by muting system audio and providing cues during recording sessions.
- Audio Input Selectors - Allows users to select specific microphone hardware for consistent audio capture.
- Speech-to-Text Normalizers - Formats output by converting spoken numbers into digits and applying punctuation and capitalization.
Star history
How this analysis was created: This summary and feature list are AI-generated from collected project material and can contain mistakes. Stars, license and language are imported from GitHub. Inclusion does not mean that we have tested or audited this project. Check the source documentation for any feature you depend on. Learn more on our About page.
