idootop/mi-gptArchived
Mi Gpt
mi-gpt is a voice assistant bridge and agent orchestrator that connects smart speakers to large language models. It functions as an integration layer that routes audio requests from hardware speakers to AI providers and converts generated text back into speech via a customizable synthesis system.
The project features a retrieval-augmented generation knowledge base that uses embeddings and external documents to provide context-aware responses. It includes a persona definition system for configuring behavioral rules, system prompts, and roleplay characteristics, alongside a plugin architecture for extending specialized behaviors and tool execution.
The system covers agent-based home automation for executing tasks via voice commands, continuous dialogue management to maintain session context, and streaming text-to-speech to reduce audio latency. It also includes utilities for conversation history backups and network routing through SOCKS proxies.
The application is distributed as container images compatible with multiple CPU architectures.
Features
- Text-to-Speech - Synthesizes generated text into spoken audio via a voice provider for playback on smart speakers.
- Smart Speaker AI Integrations - Connects smart speakers to large language models to replace default voice assistants with custom AI intelligence.
- Voice Request Routers - Routes voice requests from smart speakers to large language models to enable advanced conversational intelligence.
- Hybrid Short-and-Long Term Memory - Implements a memory architecture that integrates immediate session context with persistent long-term storage for coherent dialogues.
- Vector Knowledge Bases - Utilizes vector embeddings and external documents to inject factual data into the prompt context.
- Knowledge Base Retrieval - Uses embeddings and external documents to provide responses based on wikis and historical data.
- Voice Interaction Management - Coordinates the loop of polling device conversations and forwarding them to a language model for response generation.
- Conversation Memory Stores - Stores previous interactions to provide context-aware responses and recall details from earlier conversations.
- Persona Configurations - Provides a system for defining AI personalities, behavioral rules, and tone through specific persona configurations.
- Dialogue Context Management - Tracks interaction history and session modes to maintain context across multiple conversational turns.
- LLM Provider Integrations - Implements connectivity and authentication adapters to link the system to external large language model providers.
- LLM Voice Bridges - Bridges smart speakers to LLM APIs to enable advanced conversational intelligence and voice control.
- Persona Definitions - Defines identity, behavior rules, and prompt templates to personalize interaction dynamics.
- System Prompt Configurations - Allows the definition of behavioral instructions and roles via system prompt configurations to guide AI responses.
- RAG Context Retrieval - Implements a retrieval-augmented generation system using embeddings and documents to provide context-aware responses.
- Text-to-Speech Integrations - Sends generated text incrementally to speaker hardware to reduce audio latency.
- Assistant Personalization - Enables the creation of personalized voice assistants with custom personas, system prompts, and behavioral rules.
- AI Agent Orchestration - Provides a mechanism to coordinate AI agents and tools for executing complex home automation tasks.
- AI Agent Orchestrators - Coordinates AI agents and smart home devices to execute tasks based on natural language intent.
- Smart Speaker Integration Layers - Routes audio requests from hardware speakers to large language models and text-to-speech engines.
- Intent-to-Skill Mappings - Maps natural language user intent to specific smart home devices or internet tools for task execution.
- Conversational Flow Controllers - Manages how the assistant is triggered via keywords to handle single requests or continuous modes.
- Voice-Activated Triggers - Allows switching the active speech voice profile using specific keywords detected in the audio stream.
- Persona Assignments - Updates the agent's identity and behavioral characteristics using natural language commands during active sessions.
- Conversation State Managers - Allows controlling dialogue modes and clearing conversation context through voice commands.
- Model Response Streaming - Sends text to the speaker incrementally so audio playback begins immediately during response generation.
- Conversational Dialogue Systems - Manages speaker states and response streaming to keep the microphone active between conversation turns.
- Speech Synthesis Customizations - Allows replacing default system voices with high-quality models to customize the assistant's auditory profile.
- Interaction Triggers - Enables the customization of trigger keywords and audio prompts to manage assistant activation and exit sequences.
- Web Data Retrievers - Retrieves real-time information from the internet to provide current data within AI-generated responses.
- Plugin Systems - Provides a plugin system to extend the assistant's capabilities and specialized behaviors through modular extensions.
- Voice Customization - Routes text-to-speech processing through external providers to customize the auditory profile of the voice assistant.
- Home Automation - Integrates large language models with smart home devices to execute complex automation tasks via voice commands.
- Embedded Wake-Word Detections - Supports the configuration of custom wake words and audio alerts to trigger assistant interactions on hardware.
- AI Provider Gateways - Routes conversational requests and speech synthesis tasks to third-party providers via configurable endpoints.
- Modular Plugin Extensions - Integrates specialized behaviors and tool execution capabilities through a modular plugin system.
- Speech Delivery Optimization - Controls whether output is sent incrementally or as a complete sentence to improve speaker delivery speed.
- ChatGPT 项目 - Listed in the “ChatGPT 项目” section of the Great Open Source Project awesome list.
