leon-ai/leon
Leon
Leon is a framework for building personal AI assistants that integrates large language models with local tool execution and persistent memory. It functions as an agentic workflow orchestrator and modular skill engine, enabling the creation of autonomous assistants capable of planning and executing multi-step tasks.
The system features a retrieval-augmented generation memory architecture that indexes conversation history and user facts for context-aware grounding. It utilizes a modular skill system to interact with external binaries and APIs, supported by a loop that handles tool calling, schema validation, and failure recovery.
The project covers several broad capability areas, including voice interaction through speech-to-text and text-to-speech synthesis, natural language understanding for intent parsing, and a dynamic persona engine that adapts communication tone. It also includes administrative interfaces for assistant information management and security layers for HTTP API and client socket access.
The application is provided as a dockerized AI server to ensure consistent deployment and hosting.
Features
- Agentic Loops - Orchestrates autonomous cycles of observing, reasoning, and acting to execute multi-step tasks via a loop.
- Agentic Workflow Automations - Builds multi-step autonomous plans that execute external tools and binaries to complete complex goals.
- Long-term Memory Stores - Organizes user information into persistent and discussion stores to separate long-term facts from history.
- Hybrid Short-and-Long Term Memory - Organizes data into persistent, daily, and discussion stores to balance long-term facts with immediate context.
- Agentic Workflow Orchestration - Executes complex processes by delegating tasks to autonomous agents and native skills to reach defined goals.
- Modular Agent Skill Executions - Provides a modular skill system where independent resolvers and logic modules can be toggled at runtime.
- Memory and Context Systems - Implements a layered storage architecture for managing persistent user facts and conversation history to ground AI responses.
- Agentic Workflow Orchestration - Functions as an engine that plans and executes multi-step tasks using tool calling and failure recovery.
- Conversation State Management - Tracks conversational state and entities across multiple turns to maintain continuity in dialogue.
- Modular Skill Design - Provides a framework for designing modular resolvers and logic to extend assistant capabilities for specific domains.
- External Tool Execution - Enables the reliable execution of real-world binaries and external tools within autonomous workflows.
- Grounded Answer Generation - Uses real-time environment data and a private self-model to ensure responses match the user's actual setup.
- Large Language Model Integration - Integrates large language models with internal infrastructure to power natural language processing and task execution.
- AI Model Integrations - Provides adapters and interfaces for connecting both local and cloud-based AI models to the assistant framework.
- Embedding-Based Retrieval - Indexes memory into vector collections and uses reranking to retrieve the most relevant contextual facts.
- Memory Persistence - Stores durable preferences and recent conversation history to maintain context across different interactions.
- Multi-step Goal Execution - Performs complex, multi-step tasks by independently planning and carrying out sequences of actions.
- Environment State Grounding - Combines real-time environment state with indexed memory and reranking to ensure context-aware responses.
- Personal AI Assistants - Creates a private digital assistant that manages schedules, memories, and daily tasks using local or remote models.
- Task Planning Systems - Decomposes complex objectives into actionable tool workflows and coordinates their execution.
- Modular Skill Authoring - Provides a framework for developing modular resolvers and logic to extend the assistant's capabilities.
- Intent-to-Skill Mappings - Uses model-based parsing to map natural language user utterances to specific modular skills and tools.
- Deterministic-Agentic Hybrid Planning - Toggles between deterministic workflow execution and dynamic agent-led planning based on goal complexity.
- Assistant Management - Provides interfaces and server-side logic for managing the operational lifecycle and data of the personal assistant.
- Voice Interaction Management - Manages wake-word detection, speech recognition, and text-to-speech synthesis to enable hands-free operation.
- Operational Mode Instructions - Allows toggling between predictable execution, dynamic planning, and hybrid operational modes based on task needs.
- Behavioral State Management - Implements a private self-model and mood system that dynamically adapts the assistant's communication tone and task generation.
- AI Knowledge Management - Organizes personal data into layered memory stores to ground AI responses in real-time and historical context.
- Runtime Environment Tracking - Tracks system, network, and environment state through dedicated files to prioritize real-time data.
- Conversational Tone Adaptation - Dynamically adjusts the assistant's communication style and humor based on time and environmental signals.
- Argument Repairers - Validates tool schemas and repairs malformed model-generated arguments using failure-aware retries.
- Conversation Slot Filling - Tracks missing information in dialogues and prompts the user for specific entities until requirements are met.
- Dynamic Mood Systems - Implements a dynamic persona engine and mood system to adapt the tone of generated responses.
- Speech-to-Text Conversions - Transforms audio input into processable text using integrated speech-to-text solutions.
- Voice Interfaces - Integrates speech-to-text, text-to-speech, and wake-word detection to enable hands-free voice operation.
- Voice Controlled Interfaces - Integrates speech-to-text and text-to-speech technologies to enable hands-free interaction with the AI system.
- Task Automation Frameworks - Orchestrates custom logic and third-party integrations to automate specific actions via modular components.
- Socket Communication - Utilizes persistent socket connections for real-time event distribution between the AI server and clients.
- Human-in-the-Loop Workflows - Pauses automated execution to request missing information from the user before resuming the workflow.
- Agnostic Message Buses - Implements a real-time message bus to decouple core AI logic from various client-side user interfaces.
- Public API Exposure - Exposes internal modules and query capabilities remotely through a secure, standard web-based HTTP API.
- AI Chatbots - Personal assistant for task automation and interaction.
- Artificial Intelligence - Open-source personal assistant for task automation.
- Automation - Open-source personal assistant capable of running on your server.
- Automation Tools - Listed in the “Automation Tools” section of the Awesome Selfhosted awesome list.
