wzpan/wukong-robot
Wukong Robot
wukong-robot is an open-source, Chinese-language voice assistant platform that integrates ChatGPT for multi-turn conversational AI. It is built around a plugin-based smart speaker framework, combining offline wake word detection with local speech synthesis to enable hands-free, voice-controlled interactions without requiring a constant internet connection.
The platform distinguishes itself through its modular architecture, supporting custom wake word training via the command line and a plugin system that routes user intents using regular expressions for extensible functionality. It offers multi-engine speech synthesis, including local, cloud-based, and cloned voice options, and can stream ChatGPT responses piece by piece to reduce perceived wait time. Hardware button integration and LRU cache eviction further enhance the user experience and system efficiency.
Beyond its core identity, wukong-robot provides capabilities for smart home voice control, including music playback control, setting recurring reminders, and playing remote audio streams. It also supports local AI question answering, speech transcription via Whisper, and the ability to configure an OpenAI proxy for network-restricted environments. The project includes experimental integration with brain-computer interfaces for alternative input methods.
Documentation and installation guidance are available through the project's repository, covering setup for the voice assistant platform, plugin development, and configuration of the various speech and AI backends.
Features
- Voice Assistants - Builds a Chinese-language voice-controlled smart speaker with local wake word detection and multi-turn conversational AI.
- Wake Word Detection - Detects a custom wake word locally using a lightweight engine for reliable hands-free activation.
- Wake Word Detection Models - Detects a custom wake word locally using a lightweight engine for hands-free activation.
- CLI-Based Trainings - Generates custom wake word models from the command line for offline voice assistant activation.
- CLI-Based Trainings - Generates custom wake word models from the command line for offline voice assistant activation.
- Local Speech Synthesis - Generates speech using a local text-to-speech engine that supports custom voice recordings.
- Chinese-Language Platforms - An open-source voice-controlled smart speaker platform supporting ChatGPT multi-turn dialogue and offline wake word detection.
- Multi-Voice Synthesis Engines - Supports multiple text-to-speech backends including local, cloud, and cloned voice engines.
- Embedded Wake-Word Detections - A lightweight keyword spotting engine that detects custom wake words locally without internet.
- Plugin-Based Frameworks - A modular smart speaker platform that extends functionality through plugins for music, reminders, and custom commands.
- ChatGPT - Enables a conversational AI that keeps context across multiple turns of dialogue using ChatGPT.
- Voice Interfaces - Integrates ChatGPT for natural multi-turn dialogue with speech synthesis and wake word activation.
- Intent-Driven Plugin Executors - Routes user commands through a plugin system that matches intents using regular expressions.
- ChatGPT API Access - Sends ChatGPT replies to the user piece by piece as they are generated, reducing perceived wait time.
- Voice Cloning Synthesizers - Uses a voice model trained on a dataset to generate speech that mimics a specific person's voice.
- LLM Response Streaming - Sends ChatGPT replies to the user piece by piece as they are generated, reducing perceived wait time.
- Automatic Speech Recognition - Converts spoken audio to text using a cloud-based automatic speech recognition service.
- Local Question Answerers - Answers user questions using a locally-run engine for fast, customizable question answering.
- Recurring Reminders - Schedules timed or periodic reminders that trigger a spoken notification at the scheduled moment.
- Music Playback - Pauses, plays, skips tracks, and adjusts volume for a music player via voice commands.
- Remote Audio Streaming - Plays audio from a network URL to stream content directly from music and sound plugins.
- Cloud Speech Synthesizers - Generates speech using a cloud-based text-to-speech service for natural-sounding voice output.
- Voice Control Integrations - Controls music playback, sets reminders, and triggers actions via voice commands on supported hardware.
- Regex-Based Match Rules - Defines custom regular expression patterns to trigger any installed plugin for intent routing.
- Hardware Button Customizations - Assigns custom functions to device hardware buttons for triggering wake and other actions.
