1rgs/claude-code-proxy
Claude Code Proxy
This project is an LLM API proxy gateway and compatibility layer designed to route, translate, and proxy requests between model clients and various large language model providers. It functions as a multi-provider router that maps model requests to alternative backends based on configurable tiers and capabilities.
The gateway acts as a translation layer that converts API request and response formats between different providers, such as OpenAI or Gemini, to ensure compatibility. It includes a tool-use proxy to handle the execution and processing of model tool definitions and function calls, and it supports an interface for translating and forwarding incremental streaming responses to reduce latency.
The system provides model provider abstraction, allowing the switching of backends without changing client application code. It manages conversation context tracking and utilizes a model-to-model mapping table to redirect requests based on model size and capability.
Features
- LLM Gateways - Functions as a centralized interface that aggregates multiple LLM providers into a single, translated API.
- Provider-Agnostic LLM Routing - Implements a gateway that routes requests to various LLM providers based on configurable tiers and capabilities independently of the provider API.
- API Translation Layers - Converts API calls from one provider's format into another compatible format via a dedicated translation layer.
- Model Compatibility Proxies - Provides a proxy layer that standardizes request and response formats between different proprietary LLM providers to ensure compatibility.
- Model Provider Abstractions - Provides a normalized interface that allows switching between AI providers without changing application code.
- Request-Response Translation Layers - Converts incoming API calls and outgoing responses between different provider schemas to maintain client compatibility.
- Response Translation Layers - Converts model outputs from various providers back into a standardized API response structure.
- Tool-Execution Loops - Orchestrates the request-response cycle between an AI model's tool request and the external execution result.
- Tool-Use Proxies - Provides a proxy to handle the execution and processing of model tool definitions and function calls.
- LLM Tool Proxies - Provides a middleman server that handles the execution and processing of model tool definitions and function calls.
- API Proxies - Routes requests from compatible clients to different model providers using a translation middleware.
- Conversation History Managers - Tracks and sequences message history to maintain dialogue state for stateful AI interactions.
- Conversation Context Tracking - Maintains a sequential history of messages to preserve dialogue state across multiple request cycles.
- Model Tier Selectors - Redirects requests to different model sizes or tiers based on a predefined configuration mapping.
- Streaming Interfaces - Ships an interface for translating and forwarding incremental model responses to clients to reduce perceived latency.
- LLM Response Streaming - Implements incremental delivery of model outputs to reduce perceived latency for the end user.
- Model Identifier Mappings - Redirects specific model requests to alternative provider backends based on a configurable mapping of capabilities.
- LLM Backend Mappings - Routes requests to alternative provider backends based on a configurable mapping of model capabilities and sizes.
- Real-Time Text Streaming - Implements low-latency incremental response delivery for AI chat interfaces.
- Token Stream Relays - Forwards incremental model outputs to the client in real time to minimize perceived latency.
