GLaDOS

GLaDOS is a multimodal AI agent framework designed to create autonomous systems that process text, speech, and visual data to interact with users and their environment. It centers on an AI personality framework that emulates complex character personas using a multi-agent architecture and configurable behavioral profiles.

The project distinguishes itself through an integrated tool layer that connects language models to external hardware, smart home devices, and system APIs via a standardized protocol. It features a character text-to-speech engine with low-latency playback and interruption handling, alongside a memory and state manager that tracks reactive emotional states and persists long-term facts to maintain conversational consistency.

The system encompasses a wide range of capabilities, including vision-language perception for environmental understanding and state-based proactive triggering for autonomous action execution. It also implements a constitutional behavior layer to monitor and adjust agent outputs in real-time, ensuring adherence to predefined personality traits and guidelines.

The system includes a terminal control interface for managing speech recognition, voice settings, and status panels.

Features

Persona Emulation - Implements runtime response generation based on complex character profiles to emulate specific fictional personas.
AI Companion Personality Frameworks - Emulates complex character personas using a multi-agent architecture and configurable behavioral profiles.
Long-term Memory Stores - Persists user preferences and conversation summaries in long-term memory to maintain consistency across sessions.
Multimodal Context Providers - Merges real-time visual, speech, and text data into a unified context for environmental reasoning.
Proactive Agency Implementations - Develops systems capable of initiating proactive behaviors and executing external tools without requiring direct user prompts.
Language Model Integrations - Provides adapters and interfaces to connect the system to various cloud or local language model providers.
LLM Tooling Integrations - Implements a standardized protocol for connecting language models to external hardware, smart home devices, and system APIs.
Conversation Memory Managers - Manages long-term context and user-specific facts across multiple sessions using persistent storage and summarization.
Conversational State Managers - Tracks reactive emotional states and persists long-term facts to ensure conversational consistency.
Emotional State Mapping - Maps internal emotional states and personality traits to specific communication tones and expressions.
External Tool Integration - Enables the AI agent to interact with external APIs, hardware, and smart home devices through a standardized protocol.
Dynamic Behavior Modeling - Maintains consistent character behavior by tracking emotional states and persisting user-specific facts in long-term memory.
Dynamic Mood Systems - Tracks reactive emotional states and personality traits to dynamically influence the tone of generated responses.
Text-to-Speech Conversions - Generates audible spoken responses using a variety of regional accents and gender-specific voice profiles.
Multimodal Voice Integrations - Combines vision, speech, and text inputs into a single reasoning loop for environmental perception and response.
Speech Interruption Management - Features low-latency speech synthesis that immediately halts audio playback when user voice activity is detected.
Tool-Protocol Standardizations - Uses a standardized protocol to communicate between language models and external smart home or system APIs.
Visual Input Processing - Processes real-time visual data using vision-language models to understand and respond to the environment.
Character-Conditioned Voice Synthesizers - Provides a speech synthesis system that generates audio mimicking the specific tone and cadence of target personas.
Autonomous AI Agents - An autonomous system that processes text, speech, and visual data to interact with users and the environment.
Persona-Specific Speech Engines - Provides a voice synthesis system that converts model outputs into persona-specific audio with low-latency playback.
Multimodal Input Processors - Converts speech, text, and visual data into a unified tensor context for reasoning and environmental understanding.
Speech Interruption Handlers - Detects when a user speaks over the agent and immediately stops audio playback to listen to new input.
Model Context Protocol Integrations - Implements the Model Context Protocol to connect the agent to external protocol servers and smart home tools.
Autonomous Agent Execution - Enables the system to independently initiate proactive behaviors and execute tasks without needing a direct user prompt.
Conversation History Management - Stores thread-safe dialogue and employs summarization to compress conversation history when token limits are reached.
Behavioral Guideline Configuration - Adjusts model output in real-time using behavioral guidelines to ensure adherence to the AI's persona.
Real-time Output Filtering - Implements a constitutional behavior layer that monitors and adjusts agent outputs in real-time for personality consistency.
Multi-Agent Orchestrators - Coordinates specialized agents for vision and planning to create a unified and consistent character identity.
Multi-Agent Persona Simulations - Utilizes a multi-agent architecture combining vision, memory, and planning agents to generate a distinct personality.
Programmatic Speech Triggers - Triggers spontaneous agent voice responses based on internal states and sensor data.
Text-to-Speech Integrations - Converts AI-generated text into audible speech with low latency and integrated support for voice interruptions.
Ultra-Low Latency Speech Transcription and Generation - Uses an optimized model to convert text to audio with ultra-low latency to maintain natural conversational flow.
Smart Home Automation - Links large language models to external hardware and protocol servers to automate smart home devices.
Proactive Agent Triggers - Monitors sensors and internal states to spontaneously initiate speech and autonomous actions without user prompts.
AI Agent Behavior Monitors - Tracks system actions and self-adjusts agent behavior within predefined bounds using a constitutional observer.

awslabs/agent-squad

7,663View on GitHub

Agent Squad is a multi-agent system orchestrator and language model agent orchestration framework. It serves as an AI workflow automation engine and tool integration layer designed to coordinate teams of specialized agents to solve complex tasks through routing, parallel execution, and state management. The project is distinguished by its ability to dynamically compose purpose-specific agents on-demand and route requests based on intent, language, or domain expertise. It supports advanced coordination patterns, including parallel subtask distribution, sequential task pipelines, and the abilit

JetBrains/koog

3,735View on GitHub

Koog is an LLM agent framework used to build autonomous entities that execute tool-based workflows. It utilizes a graph-based workflow engine to define agent behaviors and decision paths as a directed graph of nodes and edges. The framework distinguishes itself through a model provider orchestrator that enables dynamic switching, load balancing, and automatic fallbacks between different AI backends. It implements the Model Context Protocol to connect agents to remote tool servers and features a RAG memory system using vector embeddings to maintain long-term conversation context. The project

livekit/agents

9,379View on GitHub

This project is a framework for developing multimodal AI agents that function as programmable participants in real-time communication rooms. It enables the construction of agents that can see, hear, and speak by integrating speech-to-text, large language models, and text-to-speech pipelines to facilitate low-latency, natural conversations. The system is distinguished by its advanced orchestration of real-time media and conversational flow, including support for full-duplex speech, preemptive response generation, and sophisticated interruption management. It further differentiates itself throu

MervinPraison/PraisonAI

5,592View on GitHub

PraisonAI is an autonomous AI agent platform that coordinates multiple LLM-powered agents for research, planning, and execution of complex workflows. It functions as a multi-agent orchestration framework, a workflow builder, and a Model Context Protocol server, while also providing retrieval-augmented generation through vector knowledge bases. Agents can interact via CLI, web, or standardized protocols with sandboxed code execution. The platform distinguishes itself with a rich set of agent communication protocols, including A2A, REST, WebSocket, voice and telephony integration, and MCP, allo

dnhkngGLaDOS

Features