Mi Gpt

mi-gpt is a voice assistant bridge and agent orchestrator that connects smart speakers to large language models. It functions as an integration layer that routes audio requests from hardware speakers to AI providers and converts generated text back into speech via a customizable synthesis system.

The project features a retrieval-augmented generation knowledge base that uses embeddings and external documents to provide context-aware responses. It includes a persona definition system for configuring behavioral rules, system prompts, and roleplay characteristics, alongside a plugin architecture for extending specialized behaviors and tool execution.

The system covers agent-based home automation for executing tasks via voice commands, continuous dialogue management to maintain session context, and streaming text-to-speech to reduce audio latency. It also includes utilities for conversation history backups and network routing through SOCKS proxies.

The application is distributed as container images compatible with multiple CPU architectures.

Features

Text-to-Speech - Synthesizes generated text into spoken audio via a voice provider for playback on smart speakers.
Smart Speaker AI Integrations - Connects smart speakers to large language models to replace default voice assistants with custom AI intelligence.
Voice Request Routers - Routes voice requests from smart speakers to large language models to enable advanced conversational intelligence.
Hybrid Short-and-Long Term Memory - Implements a memory architecture that integrates immediate session context with persistent long-term storage for coherent dialogues.
Vector Knowledge Bases - Utilizes vector embeddings and external documents to inject factual data into the prompt context.
Knowledge Base Retrieval - Uses embeddings and external documents to provide responses based on wikis and historical data.
Voice Interaction Management - Coordinates the loop of polling device conversations and forwarding them to a language model for response generation.
Conversation Memory Stores - Stores previous interactions to provide context-aware responses and recall details from earlier conversations.
Persona Configurations - Provides a system for defining AI personalities, behavioral rules, and tone through specific persona configurations.
Dialogue Context Management - Tracks interaction history and session modes to maintain context across multiple conversational turns.
LLM Provider Integrations - Implements connectivity and authentication adapters to link the system to external large language model providers.
LLM Voice Bridges - Bridges smart speakers to LLM APIs to enable advanced conversational intelligence and voice control.
Persona Definitions - Defines identity, behavior rules, and prompt templates to personalize interaction dynamics.
System Prompt Configurations - Allows the definition of behavioral instructions and roles via system prompt configurations to guide AI responses.
RAG Context Retrieval - Implements a retrieval-augmented generation system using embeddings and documents to provide context-aware responses.
Text-to-Speech Integrations - Sends generated text incrementally to speaker hardware to reduce audio latency.
Assistant Personalization - Enables the creation of personalized voice assistants with custom personas, system prompts, and behavioral rules.
AI Agent Orchestration - Provides a mechanism to coordinate AI agents and tools for executing complex home automation tasks.
AI Agent Orchestrators - Coordinates AI agents and smart home devices to execute tasks based on natural language intent.
Smart Speaker Integration Layers - Routes audio requests from hardware speakers to large language models and text-to-speech engines.
Intent-to-Skill Mappings - Maps natural language user intent to specific smart home devices or internet tools for task execution.
Conversational Flow Controllers - Manages how the assistant is triggered via keywords to handle single requests or continuous modes.
Voice-Activated Triggers - Allows switching the active speech voice profile using specific keywords detected in the audio stream.
Persona Assignments - Updates the agent's identity and behavioral characteristics using natural language commands during active sessions.
Conversation State Managers - Allows controlling dialogue modes and clearing conversation context through voice commands.
Model Response Streaming - Sends text to the speaker incrementally so audio playback begins immediately during response generation.
Conversational Dialogue Systems - Manages speaker states and response streaming to keep the microphone active between conversation turns.
Speech Synthesis Customizations - Allows replacing default system voices with high-quality models to customize the assistant's auditory profile.
Interaction Triggers - Enables the customization of trigger keywords and audio prompts to manage assistant activation and exit sequences.
Web Data Retrievers - Retrieves real-time information from the internet to provide current data within AI-generated responses.
Plugin Systems - Provides a plugin system to extend the assistant's capabilities and specialized behaviors through modular extensions.
Voice Customization - Routes text-to-speech processing through external providers to customize the auditory profile of the voice assistant.
Home Automation - Integrates large language models with smart home devices to execute complex automation tasks via voice commands.
Embedded Wake-Word Detections - Supports the configuration of custom wake words and audio alerts to trigger assistant interactions on hardware.
AI Provider Gateways - Routes conversational requests and speech synthesis tasks to third-party providers via configurable endpoints.
Modular Plugin Extensions - Integrates specialized behaviors and tool execution capabilities through a modular plugin system.
Speech Delivery Optimization - Controls whether output is sent incrementally or as a complete sentence to improve speaker delivery speed.

GetStream/Vision-Agents

6,029View on GitHub

strands-agents/sdk-python

6,176View on GitHub

This is an open-source Python SDK for building and orchestrating production-grade AI agents. It provides a unified framework for creating conversational agents that can use tools, maintain state, and coordinate across multiple language model providers including OpenAI, Anthropic, Google, Amazon Bedrock, and locally-hosted models. The SDK supports multi-agent orchestration through graphs, teams, and swarms, allowing several specialized agents to collaborate on complex tasks. Agents can be composed as callable tools that other agents invoke, and the framework includes policy handlers that inspe

l15y/wenda

6,173View on GitHub

Wenda is a self-hosted infrastructure and gateway platform for deploying language models within internal networks to ensure data privacy and security. It functions as a centralized hub and API gateway that unifies communication between various offline model runners and online service providers through a single interface. The platform includes a workflow orchestrator that uses custom scripts and API calls to automate complex conversation flows and model settings. It also incorporates a retrieval system that augments model responses with external knowledge retrieved from vector databases and se

AAswordman/Operit

3,373View on GitHub

Operit is a private, voice-enabled AI agent designed to run quantized large language models offline within mobile Linux environments. It functions as a plugin-based agent that combines local inference with a hands-free interaction pipeline. The system distinguishes itself through the use of role cards to manage distinct AI personas and conversation histories. It integrates a voice-driven interface utilizing speech-to-text and text-to-speech modules, and it enables device automation by dispatching shell commands and accessibility services to navigate user interfaces. The project further cover

idootopmi-gptArchived

Features

Open-source alternatives to Mi Gpt

GetStream/Vision-Agents

strands-agents/sdk-python

l15y/wenda

AAswordman/Operit

Star history

Open-source alternatives to Mi Gpt

GetStream/Vision-Agents

strands-agents/sdk-python

l15y/wenda

AAswordman/Operit