nguyenphutrong/quotio
Quotio
Quotio is a local LLM API proxy gateway and credential manager that intercepts and routes requests from command-line tools and integrated development environments to various AI model providers. It serves as a centralized authentication hub, managing API keys and service accounts to provide a unified interface for external AI agents.
The project distinguishes itself through a routing engine that implements priority-chain and round-robin load balancing to distribute workloads across multiple accounts. It features automated API key failover, which redirects requests to backup authentication keys when a primary provider returns rate limit or quota errors.
The system provides comprehensive monitoring and observability via a real-time dashboard that aggregates token consumption, costs, and success rates. It includes utilities for normalized usage tracking, account quota management, and resource monitoring specifically for IDEs. Security is handled through the use of the system-level encrypted keychain for credential storage and restricting proxy traffic to the local loopback address.
Features
- AI Request Routing - Provides centralized routing and telemetry to distribute requests across multiple AI model providers.
- Local Loopback - Implements a local proxy server that binds to loopback interfaces to intercept and route AI API requests.
- API Credential Managers - Provides a centralized, secure store for managing API keys and service accounts across multiple AI providers.
- AI Request Routers - Acts as a gateway for directing AI model requests across various providers using load balancing logic.
- Failover Strategies - Redirects API requests to healthy backup accounts when the primary agent returns rate limits or server errors.
- Model Provider Integrations - Provides connectors for linking multiple cloud-hosted AI model providers to centralize access.
- Failover Mechanisms - Automatically switches to backup AI model providers when the primary service experiences outages or quota limits.
- Usage Quotas - Centralizes the monitoring and display of spending and usage quotas across multiple AI provider keys.
- API Key Failovers - Automatically redirects requests to backup authentication keys when primary providers return rate limit or quota errors.
- AI Model Load Balancers - Distributes AI workloads across a sequence of accounts using priority-chains to optimize spending and avoid limits.
- API Traffic Distribution - Distributes outgoing API requests using priority chains, round-robin rotation, or cost-optimization strategies.
- AI Provider Proxies - Acts as a local proxy server that intercepts and routes requests from CLI tools and IDEs to LLM providers.
- Authenticated Request Proxies - Forwards requests to external AI APIs while injecting stored credentials so the calling tool does not manage tokens.
- API Key Load Balancers - Distributes AI requests across multiple API keys using round-robin or priority strategies to prevent rate limiting.
- CLI Session Proxying - Forwards and routes traffic from command-line interfaces to specific AI model providers via a local proxy.
- Automated Key Failovers - Implements automatic switching to backup authentication keys when primary providers return rate limit or quota errors.
- Credential Security Managers - Secures sensitive API keys using the system-level encrypted keychain and restricts traffic to the local loopback address.
- Keychain Storage - Secures sensitive API keys and OAuth tokens using the system-level encrypted keychain.
- API Key Authentication - Centralizes the management of OAuth flows and API keys to simplify authentication for external AI tools.
- Credential Management Tools - Provides a secure location for storing and managing multiple AI service provider keys.
- Request Interception Utilities - Intercepts outgoing network requests from CLI tools to apply centralized authentication and failover logic.
- Cost and Token Trackers - Collects and reports normalized token consumption and cost metrics across multiple AI providers.
- Provider-Agnostic LLM Routing - Routes requests to various AI backends regardless of the specific provider API using cost-optimization strategies.
- AI Coding Assistant Configurations - Automates environment and configuration profiles to route AI coding tools through a local proxy.
- Heuristic Token Estimators - Approximates token counts from character lengths or response headers for providers without real-time billing APIs.
- Service Quota Managers - Tracks available quotas across various accounts and implements automatic failover to distribute load.
- Interface Usage Aggregators - Aggregates cost and usage data across different interfaces, including command line tools and editors, into a unified view.
- Proxy Configuration Tools - Includes utilities for configuring command-line agents to route their traffic through a local proxy server.
- Unified NLP Interfaces - Simulates a standard API interface that wraps multiple AI providers into a single, intuitive entry point.
- AI Usage - Provides a dashboard for tracking real-time token consumption, costs, and success rates across AI accounts.
- Real-Time Monitoring Dashboards - Features a real-time dashboard monitoring request traffic, token consumption, and success rates with per-account breakdowns.
- IDE Resource Monitors - Monitors the consumption of AI resources specifically within supported integrated development environments.
- System Health Monitors - Provides real-time visibility into server health, quota overviews, and service alerts via a system menu bar.
- Usage Limiters - Implements spending quotas and rate limit enforcement to prevent account overages and service abuse.
- Normalized Usage Metrics - Normalizes usage limits, time windows, and metrics across different providers into a standard model to monitor spending.
- Context Usage Monitors - Tracks and visualizes token consumption and request activity within IDEs through a central dashboard.
