LLM Token Usage and Cost Trackers

Manifest is a language model provider unification system that standardizes access to multiple AI backends through a single interface. It functions as a centralized management layer for integrating various cloud-based and local model providers to simplify how applications request completions. The system provides intelligent model routing and high availability infrastructure by directing queries based on complexity and automatically triggering model fallbacks when a primary provider fails. It distinguishes itself through multi-tenant AI management, organizing agents into isolated groups with dedicated keys for authentication and telemetry. The project covers AI cost management and observability by tracking token usage, monitoring expenditures per request, and enforcing budget limits. These capabilities are supported by daily synchronization of model pricing from external sources and the tracking of performance metrics across agents. The system can be deployed as a containerized image using Docker to simplify self-hosted administration.

This project is a command-line utility designed to monitor and analyze token consumption and financial expenditure for AI coding assistants. By parsing local session logs directly on the user's machine, it provides a privacy-focused way to track development activity without transmitting sensitive data to external servers. The tool distinguishes itself through its ability to aggregate disparate log formats from multiple coding assistants into a unified, schema-agnostic representation. It features a decoupled pricing engine that allows users to apply custom model-specific cost multipliers, override default pricing, and account for different service tiers. This enables granular reporting across various dimensions, including individual interaction sessions, specific projects, or custom time-based billing windows. Beyond core tracking, the utility supports a wide range of analytical capabilities such as trend visualization, currency conversion, and the ability to inspect individual conversation logs. Users can configure reporting parameters, define project aliases, and export findings into machine-readable formats for further integration. The entire analysis process operates locally, ensuring that usage telemetry remains private and accessible even without an active network connection.

One API is a centralized gateway and orchestration platform designed to consolidate multiple artificial intelligence model providers into a single, standardized interface. It functions as a reverse proxy that intercepts incoming API requests and routes them to various third-party services, abstracting the underlying provider credentials through a unified token management system. The platform provides comprehensive administrative tools for managing API keys, rotating credentials, and enforcing security policies across diverse service integrations. It includes a persistent database-backed system for tracking real-time usage metrics and managing user-specific credit quotas, ensuring controlled resource distribution. Users can monitor these activities and configure system parameters through a self-hosted administrative dashboard. Beyond its core routing and orchestration capabilities, the project supports flexible deployment across containerized or multi-machine environments. It also allows for visual customization of the frontend interface through a directory-based asset replacement mechanism, enabling consistent branding across the administrative dashboard.

This project is a secure intermediary proxy gateway for large language model APIs. It functions as a relay service that forwards requests to AI providers while managing service accounts and routing traffic. The service provides a compatibility layer that supports multiple endpoint formats, allowing different third-party AI clients to communicate with a single provider. It distinguishes itself through a service account management system that assigns individual proxy settings to multiple accounts to prevent IP bans and distributes traffic via load balancing to avoid rate limits. The system includes a rate limiter that restricts access based on token volume, concurrency, and custom identification keys. It monitors usage through a tracking system that records token consumption and request metrics per user. Reliability is maintained through a circuit-breaker mechanism that detects upstream connection failures and pauses routing to affected accounts using cooldown timers.

Moltworker is an AI agent sandbox and model orchestrator designed for the secure execution of untrusted code and shell commands generated by large language models. It functions as a gateway proxy that routes requests to multiple AI providers through a unified interface, integrating a container runtime backed by S3-compatible object storage to persist state across ephemeral lifecycles. The system distinguishes itself by combining an AI model orchestrator with a headless browser controller for automated web scraping and screenshot capture. It manages the full lifecycle of AI agents, including multi-channel chat integration, consolidated billing across different providers, and expenditure limits to control operational costs. The platform provides a broad suite of capabilities for ephemeral environment hosting, including isolated build pipelines and the exposure of services via preview URLs. It incorporates security and observability tools such as token-based proxy authentication, response caching, and traffic analysis to monitor token usage and request volume. The infrastructure supports real-time interaction through a browser-based terminal interface using WebSocket streaming and monitors filesystem changes for automated build processes.

Agenta is a Prompt Ops lifecycle manager and prompt management platform that decouples prompt engineering from application code. It serves as a centralized system for developing, versioning, and deploying prompt templates and model configurations across different environments. The platform functions as an AI agent orchestrator with a visual interface for building agent workflows and connecting models to external tools. It further acts as an evaluation framework and observability tool, utilizing OpenTelemetry to capture execution traces, monitor latency, and track token costs. The system covers a broad range of capabilities including judge-based evaluation for scoring model outputs, registry-based prompt management for version control, and environment-based deployment to promote configurations through development and production stages. It also provides tools for converting production traces into test datasets and managing role-based access control for multi-tenant organizations. The platform can be installed using Docker Compose with reverse proxy options for traffic management.

ClawWork is a suite of tools designed to monitor agent finances, provide isolated execution environments, simulate economic behaviors, and benchmark performance. It functions as an autonomous agent sandbox where AI agents can run code and generate professional business deliverables. The project focuses on the financial sustainability of AI assistants through an economic simulation environment. This includes tools for tracking token expenditures and income generation, as well as simulations that analyze the trade-offs between immediate earnings and long-term skill acquisition. The system includes a real-time financial monitor and dashboard to visualize economic metrics and solvency. It also provides a benchmarking framework that uses sector-specific rubrics to score the quality of professional artifacts and technical deliverables.

HyperDX is an OpenTelemetry observability platform that provides centralized log management, distributed tracing, and a self-hosted monitoring stack. It functions as a unified system for collecting, indexing, and visualizing logs, metrics, and traces from cloud and container environments. The platform distinguishes itself with specialized tooling for large language model monitoring and session replay, allowing user interactions in the browser to be linked to backend telemetry. It employs schema-less JSON parsing to index structured logs dynamically and uses source maps to resolve minified stack traces back to original code. Its broader capabilities include full-stack instrumentation for various languages and serverless environments, automated event pattern clustering, and end-to-end request tracking. The system also features SQL-based telemetry querying, multi-channel alerting, and unified visualization dashboards. The software can be deployed as a self-hosted instance using Docker.

This project is an AI model API gateway and proxy server designed to provide a unified interface for interacting with diverse artificial intelligence service providers. It functions as a centralized middleware platform that routes, load balances, and translates API requests across multiple models, enabling developers to access text, image, audio, and video generation capabilities through a single, standardized integration. The gateway distinguishes itself through comprehensive administrative and financial controls, including event-driven usage accounting, real-time token consumption tracking, and granular role-based access control. It supports complex traffic management by distributing requests across multiple credential pools and providers to optimize throughput and bypass rate limits. Furthermore, it integrates a robust identity federation system that supports OIDC, OAuth, and hardware-backed passkeys to secure user access and manage multi-tenant environments. Beyond core routing, the platform provides extensive tooling for service maintenance, including automated health checks, model registry synchronization, and content moderation filters. It also features a complete billing and payment infrastructure, allowing administrators to manage user credit balances, process prepaid redemptions, and monitor cost structures across different model vendors. The system is designed for flexible deployment across containerized and distributed infrastructure, with administrative interfaces for auditing usage logs, managing API channels, and configuring global system parameters.

Casibase is an open-source platform that orchestrates multi-turn conversations with large language models and manages retrieval-augmented knowledge bases from a single interface. It provides a unified system for connecting to over 30 AI model providers, ingesting documents into vector embeddings for semantic search, and running autonomous agent loops that can drive a browser, search the web, execute commands, and integrate with external tools. The platform distinguishes itself by combining AI conversation management with infrastructure and application orchestration capabilities. It includes a visual workflow designer for composing multi-step pipelines, a Kubernetes blueprint orchestrator for deploying containerized applications with environment-specific customization, and a browser-based remote server gateway for managing SSH, RDP, and VNC connections. Role-based access control is enforced across routers, controllers, and UI layers, with single sign-on authentication and user-to-store data isolation. Beyond its core AI and automation features, Casibase offers infrastructure security scanning, token-aware billing with per-message cost tracking, and integration with enterprise messaging platforms for real-time AI responses. It provides an OpenAI-compatible API endpoint, client SDKs, and Swagger-generated documentation for programmatic access. The system supports multi-store knowledge isolation, cross-store vector sharing, and a centralized dashboard for monitoring system resources, deployment states, and usage activity across providers and users.

BAML is a prompt engineering framework and LLM client generator that defines AI prompts as type-safe functions. It serves as a structured data extraction tool and workflow orchestrator, transforming unstructured model responses into strongly typed objects using a custom schema language and alignment algorithms. The project distinguishes itself by using a compiler to generate language-specific boilerplate code for API communication and output parsing. It features a dedicated environment for designing complex prompt templates with conditional logic and reusable snippets, and employs genetic algorithms for automated prompt optimization based on performance benchmarks. The platform covers a broad range of capability areas, including provider-agnostic request routing with multi-stage fallback orchestration and an observability suite for token tracking and distributed tracing. It supports multimodal AI processing for images, audio, and PDFs, while providing tools for AI workflow validation and schema-driven output parsing. The system includes a command-line interface for project initialization and automated client generation, as well as IDE integration for real-time prompt testing and syntax validation.

Cognita is a retrieval augmented generation orchestration framework used to build pipelines that connect document stores and language models to provide grounded answers. It functions as a document ingestion pipeline and a vector database integrator, managing the process of loading, parsing, and indexing files into a searchable knowledge base. The system includes a language model gateway proxy that provides a unified API to interact with multiple different model providers. This routing layer decouples the application from specific vendors, allowing requests to be proxied through a provider-agnostic interface. The framework covers contextual information retrieval through similarity search and reranking to generate responses with source citations. It supports incremental document indexing to process new or updated files without re-indexing entire datasets and allows for the integration of various vector store implementations.