11 مستودعات
Tools for monitoring and tracking operational data such as token consumption, financial costs, and response latency in language model deployments.
Explore 11 awesome GitHub repositories matching artificial intelligence & ml · Language Model Observability. Refine with filters or upvote what's useful.
OpenHands is an autonomous agent framework designed for software engineering workflows. It provides a modular platform for orchestrating AI agents that reason, plan, and execute tasks within isolated, containerized development environments. By integrating with standard version control and development tools, the system enables agents to autonomously navigate codebases, implement features, and resolve issues through iterative reasoning and tool execution. The platform distinguishes itself through a model-agnostic orchestrator that connects diverse language models to a unified tool registry. It
Captures granular token consumption, latency, and financial expenditure data directly from language model objects after each request.
Opcode is a desktop interface designed for managing AI-assisted software development workflows. It provides a centralized workspace to organize interactive programming sessions, configure specialized automated agents, and maintain oversight of development tasks through a visual environment. The platform distinguishes itself by integrating version control for AI conversations, allowing developers to create checkpoints and branches to navigate, compare, and revert between different interaction states. It also functions as a client for standardized context protocols, enabling the connection of e
Provides a dashboard for tracking real-time token consumption and resource expenditure.
This project is a comprehensive framework for building AI-powered applications, providing a unified toolkit for orchestrating language models, autonomous agents, and interactive user interfaces. It serves as a central library for managing the entire lifecycle of AI interactions, from initial prompt generation and model provider abstraction to complex, multi-step reasoning and tool execution. The framework distinguishes itself through its deep integration with frontend development, specifically by enabling generative user interfaces that render dynamic components directly from model outputs. I
Simulates language model and embedding provider outputs using deterministic helpers to enable repeatable unit testing.
DevOps-Roadmap is a comprehensive educational repository and knowledge base designed to guide technical professionals through the complexities of modern software engineering. It functions as a structured curriculum and reference library, covering the full spectrum of skills required to master system architecture, infrastructure management, and cloud operations. The project distinguishes itself by bridging the gap between high-level architectural design and the practical realities of engineering leadership. It provides curated insights into distributed systems, data consistency, and scalable d
Provides specialized operational practices for monitoring and maintaining large language models in production environments.
Onyx is an enterprise-grade AI platform designed for knowledge management, search, and autonomous agent orchestration. It functions as a centralized system that aggregates unstructured organizational data, enabling secure, context-aware retrieval and interaction across internal documents and communication history. By integrating retrieval-augmented generation with multi-model orchestration, the platform provides a unified interface for teams to query internal knowledge bases and execute complex, multi-step business processes. The platform distinguishes itself through a focus on private infras
Monitors token consumption and financial costs to help manage operational expenses for language models.
TensorZero is an inference gateway and experimentation framework designed to manage the lifecycle of large language models in production environments. It functions as a central proxy that routes requests across multiple artificial intelligence providers while providing the infrastructure necessary to monitor performance, track costs, and ensure service reliability. The platform distinguishes itself by integrating a comprehensive evaluation engine and an observability pipeline directly into the request flow. It enables developers to conduct controlled experiments and A/B tests to compare diffe
Tracks inference metrics, latency, and costs to monitor performance and debug language model deployments in production.
Kreuzberg is a document extraction engine that converts PDFs, Office files, images, and over 90 other formats into clean, structured text and metadata. It is built around a compiled Rust core that can be used as a native library, a command-line tool, a REST API server, or a WebAssembly module for browser-based processing. The system is designed to run entirely on self-hosted infrastructure, with no data leaving the user's environment. What distinguishes Kreuzberg is its breadth of integration surfaces and its pipeline architecture. It exposes extraction capabilities through native bindings fo
Records token counts and estimated costs for each LLM call made during extraction.
Plano is an AI agent orchestrator and LLM gateway proxy that unifies access to multiple AI providers through a single interoperable interface. It functions as a model routing engine that decouples applications from specific vendors using semantic aliases, allowing traffic to be shifted between providers without modifying application code. The system distinguishes itself with intent-based agent routing, which directs prompts to specialized agents based on semantic analysis. It features an interceptor-based filter chain system that acts as guardrail middleware to enforce safety policies, rewrit
Provides operational tracking of token consumption, financial costs, and response latency.
ccstatusline is a CLI status bar framework and ANSI terminal UI library used to render dynamic, multi-line interfaces within terminal emulators. It functions as a customizable system for displaying real-time metrics, system data, and configuration settings through a widget-based composition model. The framework specializes in Powerline-style status lines, utilizing arrow separators and columnar alignment to create structured headers. It includes specialized dashboards for monitoring Git repository metadata, such as branch names and commit IDs, as well as LLM usage monitors that track API toke
Tracks token consumption, latency, and financial costs for LLM API integrations.
This project is a Git-based AI session tracker and context manager designed to record AI agent interactions, transcripts, and tool usage directly into Git repositories. It functions as a system for capturing and indexing the reasoning behind code changes, linking AI prompts and responses to specific code commits to preserve developer intent. The tool distinguishes itself by using Git as a primary storage layer for session metadata, utilizing shadow branches and checkpoints to track agent state without polluting the main commit log. It includes specialized capabilities for auditing AI contribu
Records the specific language model used within checkpoint metadata for auditing.
The server acts as a centralized ingestion engine designed to collect, normalize, and index distributed telemetry data. It functions as a backend processor that receives performance metrics, traces, and error logs from application agents, transforming them into structured documents for storage and analysis within search and analytics platforms. The system distinguishes itself through a high-throughput ingestion pipeline that utilizes asynchronous event processing and backpressure-aware flow control to maintain stability during traffic spikes. It employs modular, plugin-based transformation st
Tracks performance and execution flow of large language model applications to optimize latency and debug integration issues.