17 dépôts
Tools for inspecting the history of thoughts, tool calls, and observations during agentic workflows.
Distinguishing note: Focuses on the observability of the reasoning process rather than general application logging.
Explore 17 awesome GitHub repositories matching artificial intelligence & ml · Agent Execution Tracing. Refine with filters or upvote what's useful.
This project is a structured educational resource and technical guide for designing and implementing autonomous systems using large language models. It provides a comprehensive curriculum and code samples focused on agentic design patterns, autonomous development, and the creation of systems capable of planning and executing multi-step tasks. The resource details the implementation of agentic retrieval-augmented generation, where models autonomously plan and refine data searches. It covers a wide array of orchestrators and design patterns, including metacognitive reflection for self-correctin
Records complete task runs as traces to make internal reasoning and state transitions transparent.
DSPy is a declarative programming framework designed for building complex language model applications. It treats model interactions as modular, composable programs, allowing developers to define task logic through typed class schemas rather than relying on manually written prompts. By organizing workflows into hierarchical, reusable Python objects, the framework enables the construction of sophisticated AI systems that manage state and execution flow independently. The framework distinguishes itself through an automated optimization engine that iteratively refines prompt instructions and few-
Provides visibility into the step-by-step decision-making process and tool usage of autonomous agents.
This project is a Python framework for building autonomous, event-driven agent systems. It provides a unified runtime for orchestrating multi-agent workflows, managing persistent conversation state, and executing code within secure, isolated sandbox environments. The framework is designed to handle complex task delegation, allowing agents to invoke other agents as tools while maintaining context across multi-turn interactions. The framework distinguishes itself through its deep integration with the Model Context Protocol, enabling agents to connect to external data sources and remote services
Captures and exports telemetry data for agent workflows with granular control over authentication and data sensitivity.
Agentscope is a comprehensive toolkit for developing and orchestrating autonomous multi-agent systems. It provides a unified framework for building agents that can reason, execute tools, and manage memory, enabling the creation of complex, collaborative workflows where multiple specialized agents interact to solve multi-step objectives. The platform distinguishes itself through a robust orchestration engine that supports both sequential and concurrent agent pipelines. It utilizes a centralized event bus for real-time telemetry, allowing developers to track agent reasoning, tool usage, and sys
Provides real-time telemetry and visualization of agent reasoning and tool usage through a centralized event bus.
Deepagents is an LLM agent orchestration platform and stateful application server designed for deploying and managing AI agents built with computational graphs. It provides a containerized runtime environment that handles agent execution, state persistence, and the versioning of AI assistants. The platform distinguishes itself through deep integration with the Model Context Protocol, allowing agents to function as servers that expose tools and capabilities to external clients. It features a sophisticated observability suite for capturing execution traces, performing LLM-based evaluations agai
Automatically captures agent invocations, tool calls, and LLM interactions for debugging and reasoning inspection.
Forem is an open-source platform designed for building and managing technical communities. It functions as a social publishing engine that enables members to share long-form content, participate in threaded discussions, and engage through social interactions. The platform provides tools for organizations to maintain branded profiles, host community hackathons, and facilitate collaborative learning through structured educational tracks. Beyond its social features, Forem integrates advanced capabilities for AI agent workflow orchestration and codebase knowledge graphing. It allows developers to
Offers observability tools to trace agent reasoning, tool usage, and performance metrics in production.
Claude Code is a command-line interface and multi-agent orchestration framework designed for autonomous software engineering. It enables AI agents to perform codebase modifications, debugging, and Git workflow management while coordinating multiple specialized agents to decompose and execute complex engineering tasks in parallel. The system distinguishes itself through a high degree of isolation and safety, utilizing Git worktrees to create independent working directories for concurrent agents and implementing a tiered permission system that combines user rules, project policies, and OS-level
Provides tools for inspecting the history of thoughts, tool calls, and observations to diagnose agent behavior.
PydanticAI is a Python framework designed for building production-grade autonomous agents. It provides a unified interface for interacting with diverse language models, enabling developers to construct agents that perform complex tasks through structured data validation, tool execution, and multi-turn conversation management. The library centers on type-safe schema enforcement, ensuring that model inputs and outputs remain consistent and reliable throughout the agent's lifecycle. The framework distinguishes itself through a robust architecture that emphasizes modularity and testability. It ut
Captures detailed telemetry on message history, tool calls, and token usage to facilitate debugging and performance optimization of agentic workflows.
RagaAI-Catalyst is a suite of software implementation tools providing an SDK, dashboard, and platform for monitoring, debugging, red-teaming, and evaluating agentic AI workflows. It serves as an observability framework for tracing the execution paths of large language models and multi-agent systems. The project distinguishes itself through a security suite for automated red-teaming and vulnerability scanning to detect biases, alongside a centralized prompt registry that decouples templates from application code. It further provides an evaluation platform that combines synthetic data generatio
Implements a system for tracing agent reasoning, tool calls, and decision-making processes to debug complex workflows.
Memori is an AI agent memory middleware platform designed to provide persistent, context-aware recall for language models. It functions as a non-intrusive layer that intercepts outbound model requests to automatically capture interaction history and execution traces, ensuring that agents maintain continuity across sessions without requiring modifications to existing application logic. The platform distinguishes itself through a dual-model storage architecture that maintains information as both structured relational primitives for precise fact retrieval and rolling narrative summaries for situ
Intercepts agent calls to automatically persist execution traces and conversation context across sessions.
This project is a framework for managing generative AI services through a unified provider interface and adapter layer. It provides a standardized API for calling multiple cloud-based and locally hosted models, translating provider-specific parameters and responses into a uniform format. The system includes an agent orchestrator designed for long-running tasks, featuring state persistence for resuming runs and execution tracing to monitor decision-making processes. It integrates the Model Context Protocol to connect models to external servers and filesystems and employs a policy-based executi
Generates detailed trace output for agent activities to monitor the reasoning process in a visual viewer.
Open-SWE is an asynchronous software engineering agent and orchestrator designed to automate end-to-end coding tasks and pull request reviews. It functions as a middleware framework that coordinates long-running AI operations across multiple subagents, utilizing state persistence and human-in-the-loop oversight to manage complex workflows. The system is distinguished by its use of isolated remote Linux sandboxes for secure code execution and shell command processing. It features a webhook-driven integration platform that triggers automated engineering tasks via mentions and events in GitHub,
Provides tools for inspecting the history of thoughts, tool calls, and state transitions during agentic workflows.
Eino is an AI agent development kit and LLM application framework designed for building autonomous agents and orchestrating complex language model workflows. It serves as a multi-agent orchestration engine and workflow orchestrator, providing a graph-based execution model to route data between models, tools, and retrievers. The framework distinguishes itself through a robust set of multi-agent coordination patterns, including supervisor-led management, sequential flows, and autonomous reasoning loops like ReAct. It features advanced agent execution controls such as active turn preemption, che
Provides tools for inspecting the history of thoughts, tool calls, and observations during agentic workflows.
mcp-agent is a framework for building AI agents that integrate with Model Context Protocol servers to execute tools and access data. It functions as a multi-agent orchestrator and protocol-compliant server, enabling the creation of agents that can discover and invoke tools from connected external servers. The project distinguishes itself through a durable workflow engine that supports long-running tasks capable of pausing, resuming, and surviving restarts. It implements complex orchestration patterns, including iterative evaluator-optimizer loops, hierarchical workflow nesting, and specialist
Captures execution spans for routing and planning to visualize agent reasoning and tool usage in telemetry backends.
Droidrun is a mobile device automation framework that uses large language models to translate natural language commands into executable actions on mobile operating systems. It functions as an agent orchestrator and UI automation engine, providing a reasoning engine that decomposes complex mobile tasks into smaller, manageable steps. The system distinguishes itself through a hierarchical action translation process and the ability to analyze accessibility trees and screenshots to determine the visual layout and current status of mobile applications. It supports execution across both physical ha
Logs and visualizes the decision-making path of an agent to debug its operational logic.
This project is a collection of educational resources and technical guides focused on the development and implementation of large language models. It provides a comprehensive curriculum covering transformer architectures, training methods, and deployment strategies. The materials provide detailed instructions for building autonomous agents using reasoning loops and tool integration, as well as guides for fine-tuning models through supervised learning and preference optimization. It also includes tutorials for constructing retrieval augmented generation pipelines and implementing transformer m
Provides tools for inspecting the history of thoughts and tool calls during agentic workflows.
The BeeAI Framework is an LLM agent framework and multi-agent orchestration engine used to build autonomous agents that coordinate reasoning, tool execution, and complex workflows. It functions as a structured AI output controller and RAG integration library, providing a unified interface to manage multiple language model providers. The framework is distinguished by its implementation of the Model Context Protocol, allowing agents, tools, and models to be shared between different AI platforms and hosted as agentic tooling servers. It enables the design of collaborative agent teams through dec
Inspects the history of thoughts, tool calls, and observations to debug agentic workflows.