voice assistant platform / framework

wukong-robot is an open-source, Chinese-language voice assistant platform that integrates ChatGPT for multi-turn conversational AI. It is built around a plugin-based smart speaker framework, combining offline wake word detection with local speech synthesis to enable hands-free, voice-controlled interactions without requiring a constant internet connection. The platform distinguishes itself through its modular architecture, supporting custom wake word training via the command line and a plugin system that routes user intents using regular expressions for extensible functionality. It offers mul

Mycroft Core is an open-source voice assistant platform that processes spoken commands and runs modular skills for tasks like home automation and information retrieval. It is built around a cloud-paired device framework, where a voice assistant device links to a cloud account via a spoken pairing code to synchronize API keys and skills. The platform operates through an intent-parsing pipeline that processes speech recognition, intent extraction, and skill matching, all coordinated by a message bus architecture that decouples speech processing, skill execution, and audio output. A wake word en

NeMo is a multimodal AI framework and toolkit designed for the development, training, and scaling of large language models, generative AI systems, and speech-based models. It functions as an automatic speech recognition toolkit, a text-to-speech engine, and a framework for building models that process and generate combinations of text, image, and audio data. The project serves as a conversational AI orchestrator capable of managing real-time, interruptible voice interactions. It provides specialized workflows for speech translation, converting spoken audio from one language into text or speec

Pipecat is a framework and software development kit for building real-time multimodal AI agents and speech-to-speech systems. It utilizes a frame-based data pipeline to route audio, video, and text through a modular sequence of processors, enabling the orchestration of low-latency conversational AI. The project is distinguished by its ability to coordinate complex multimodal services, including speech-to-text, language models, and text-to-speech, within a single pipeline. It features semantic voice activity detection for natural turn-taking, state-machine conversation flows for dialogue manag

This project is a framework for developing multimodal AI agents that function as programmable participants in real-time communication rooms. It enables the construction of agents that can see, hear, and speak by integrating speech-to-text, large language models, and text-to-speech pipelines to facilitate low-latency, natural conversations. The system is distinguished by its advanced orchestration of real-time media and conversational flow, including support for full-duplex speech, preemptive response generation, and sophisticated interruption management. It further differentiates itself throu

LiveKit is a comprehensive framework for building and orchestrating real-time, multimodal AI agents that interact with users through voice, video, and text. It provides a centralized, event-driven architecture to manage the entire lifecycle of automated participants, from initialization and session state management to graceful shutdown. By utilizing a selective forwarding unit, the platform efficiently routes media streams between participants and agents, ensuring low-latency communication and secure, token-based authentication for all connections. The platform distinguishes itself through it

Vocode-core is a framework for building real-time conversational AI voice agents. It serves as a conversational orchestrator and pipeline that integrates speech-to-text, large language models, and text-to-speech services to enable low-latency voice interactions. The project features a provider-agnostic interface that allows for swappable speech and language model providers, including support for both cloud APIs and local binaries. It distinguishes itself through a specialized telephony integration layer that enables agents to be deployed across phone lines, WebRTC, and virtual meeting platfor

This project is a framework for building local voice assistants and a real-time audio streaming server. It functions as a containerized inference engine and a multilingual speech pipeline that orchestrates speech-to-text, language models, and text-to-speech components to convert spoken input into spoken output. The system is distinguished by its use of WebSocket-based bidirectional streaming for low-latency interactions. It features a voice activity detection system that manages speech boundaries and handles user barge-in interruptions during assistant playback. It also supports custom voice

Xiaozhi-esp32 is an open-source firmware platform designed for building voice-interactive embedded systems on resource-constrained microcontrollers. It functions as an IoT conversational device platform that manages live audio input, speech synthesis, and conversational state transitions to facilitate real-time natural language interaction. The system distinguishes itself by bridging language models with physical hardware through standardized protocols, allowing for the execution of commands on local peripherals or remote smart home services. It utilizes a specialized architecture to coordina

Snowboy is a deep neural network-based wake word detection engine that runs entirely on-device, processing live audio to recognize a spoken trigger phrase without sending any data to the cloud. The project provides both pre-trained universal models for common wake words like "Alexa" or "Snowboy" and the ability to detect a custom wake word defined by the user. The engine is designed to be lightweight enough for embedded devices, using less than 10% CPU on a single-core 700MHz ARM processor such as a Raspberry Pi. Users can train a personalized wake-word model by submitting voice samples throu

This project is a speech recognition and translation engine that utilizes a sequence-to-sequence transformer architecture to convert audio into text. It is built upon a weakly supervised learning framework, which leverages large-scale, unlabelled audio-transcript data to create generalized speech representations capable of performing simultaneous transcription, language identification, and translation. The system distinguishes itself through a unified multi-task modeling approach that shares token sequences across different objectives, allowing it to handle diverse languages and vocabularies