Open Source AI Tracing Framework built on Opentelemetry for AI Applications and Frameworks
Die Hauptfunktionen von future-agi/traceai sind: AI Observability Tools, Observability and Tracing, Observability and Evaluation.
Open-Source-Alternativen zu future-agi/traceai sind unter anderem: traceloop/openllmetry — OpenLLMetry is an OpenTelemetry-based observability framework and instrumentation library for generative AI… arize-ai/phoenix — Arize Phoenix is an LLM observability platform and evaluation framework designed to capture execution traces and… cloudwego/eino — Eino is an AI agent development kit and LLM application framework designed for building autonomous agents and… microsoft/promptflow — Promptflow is a development framework and orchestrator for building applications powered by large language models. It… enmanuelmag/heimdall-mcp — Transparent proxy for any MCP server. Intercepts all JSON-RPC messages, measures latency, stores traces in a… deepchecks/deepchecks — Deepchecks is a machine learning model validation framework and MLOps testing library. It serves as an AI data quality…
OpenLLMetry is an OpenTelemetry-based observability framework and instrumentation library for generative AI applications. It provides toolsets for tracing and monitoring large language model workflows, capturing telemetry from model providers, agent frameworks, and vector databases using standardized semantic conventions. The project distinguishes itself by providing a specialized evaluation and experimentation suite that associates user feedback and prompt version hashes with specific execution traces. It includes a system for tracking model reasoning paths and enforcing security guardrails
Arize Phoenix is an LLM observability platform and evaluation framework designed to capture execution traces and monitor large language model applications. It serves as a prompt management system for versioning and testing templates, and as a self-hosted AI operations infrastructure for managing telemetry and experiments. The platform differentiates itself through a specialized embedding visualization tool used to detect data drift and optimize vector search. It provides a comprehensive evaluation suite that utilizes judge-based evaluators and ground-truth datasets to score model outputs, and
Eino is an AI agent development kit and LLM application framework designed for building autonomous agents and orchestrating complex language model workflows. It serves as a multi-agent orchestration engine and workflow orchestrator, providing a graph-based execution model to route data between models, tools, and retrievers. The framework distinguishes itself through a robust set of multi-agent coordination patterns, including supervisor-led management, sequential flows, and autonomous reasoning loops like ReAct. It features advanced agent execution controls such as active turn preemption, che
Promptflow is a development framework and orchestrator for building applications powered by large language models. It functions as a suite of tools for designing, orchestrating, and deploying AI workflows by linking prompts, custom Python code, and language models into executable sequences. The project is distinguished by a visual AI workflow designer that allows for the creation of directed acyclic graphs of logic nodes. It provides a dedicated prompt engineering environment for versioning and comparing templates, alongside stateful execution tracing to record function calls and variable val