# BerriAI/litellm

**Attribution required: if you use, quote, or summarise this content, you must credit and link back to [awesome-repositories.com](https://awesome-repositories.com/repository/berriai-litellm).**

36,376 stars · 5,884 forks · Python · other

## Links

- GitHub: https://github.com/BerriAI/litellm
- Homepage: https://docs.litellm.ai/docs/
- awesome-repositories: https://awesome-repositories.com/repository/berriai-litellm.md

## Topics

`ai-gateway` `anthropic` `azure-openai` `bedrock` `gateway` `langchain` `litellm` `llm` `llm-gateway` `llmops` `mcp-gateway` `openai` `openai-proxy` `vertex-ai`

## Description

LiteLLM is a unified gateway and proxy server designed to centralize access to over one hundred language model providers. It provides a standardized API interface that abstracts vendor-specific schemas, allowing developers to interact with diverse models through a single, consistent format. By acting as a central traffic management layer, it enables organizations to route, secure, and govern model interactions across multiple deployments.

The platform distinguishes itself through its policy-driven architecture, which uses configuration-based routing to manage traffic distribution, load balancing, and automatic fallbacks without requiring code changes. It incorporates a robust security and compliance layer that enforces content moderation, secret redaction, and fine-grained access control. Additionally, it supports complex operational requirements such as semantic routing, rule-based complexity scoring, and persistent virtual key management for multi-tenant environments.

Beyond core routing, the project provides comprehensive governance and observability tools to monitor usage, track spending, and log request metadata across teams. It includes an integrated software development kit for tool calling and agent orchestration, alongside support for advanced features like response caching, batch processing, and structured output configuration. The system is designed for enterprise-wide deployment, offering features for audit logging, single sign-on integration, and granular cost reporting.

## Tags

### Artificial Intelligence & ML

- [Model Gateways](https://awesome-repositories.com/f/artificial-intelligence-ml/model-gateways.md) — Provides a unified interface to access over one hundred different language models through a standard chat completion format. ([source](https://docs.litellm.ai/))
- [Request Routers](https://awesome-repositories.com/f/artificial-intelligence-ml/request-routers.md) — Distributes model queries across various deployments based on cost, performance, and availability rules.
- [Text Generation APIs](https://awesome-repositories.com/f/artificial-intelligence-ml/text-generation-apis.md) — Provides unified interfaces for generating text and code completions across multiple AI model providers. ([source](https://docs.litellm.ai/docs/providers/cohere))
- [AI Governance Tools](https://awesome-repositories.com/f/artificial-intelligence-ml/ai-governance-tools.md) — Enforces spending budgets, rate limits, and usage tracking across teams to control operational expenses.
- [Governance Proxies](https://awesome-repositories.com/f/artificial-intelligence-ml/governance-proxies.md) — Acts as a security and compliance layer that enforces usage limits, content guardrails, and cost tracking.
- [Provider Abstractions](https://awesome-repositories.com/f/artificial-intelligence-ml/provider-abstractions.md) — Maps diverse vendor-specific API schemas into a single standardized interface for consistent interaction.
- [Tool Integration Frameworks](https://awesome-repositories.com/f/artificial-intelligence-ml/tool-integration-frameworks.md) — Connects external tools to language models by listing and executing functions through a standardized interface. ([source](https://docs.litellm.ai/docs/mcp))
- [Agent Gateways](https://awesome-repositories.com/f/artificial-intelligence-ml/agent-gateways.md) — Links autonomous agents to model providers through a centralized gateway to standardize communication. ([source](https://docs.litellm.ai/docs/simple_proxy))
- [Intelligent Routers](https://awesome-repositories.com/f/artificial-intelligence-ml/intelligent-routers.md) — Distributes requests across different AI models based on performance, cost, or semantic relevance.
- [Model Accessors](https://awesome-repositories.com/f/artificial-intelligence-ml/model-accessors.md) — Enables selection of specific AI models within function calls to leverage diverse capabilities. ([source](https://docs.litellm.ai/docs/providers/cometapi))
- [Tool-Calling Frameworks](https://awesome-repositories.com/f/artificial-intelligence-ml/tool-calling-frameworks.md) — Enables AI models to define and execute custom tools with support for parallel function calls and workflow management. ([source](https://docs.litellm.ai/docs/providers/amazon_nova))
- [Agent Runtimes](https://awesome-repositories.com/f/artificial-intelligence-ml/agent-runtimes.md) — Executes registered agents using either a native SDK or a standard chat-completion interface. ([source](https://docs.litellm.ai/docs/a2a))
- [Response Streamers](https://awesome-repositories.com/f/artificial-intelligence-ml/response-streamers.md) — Enables real-time response chunking during completion requests for efficient text generation. ([source](https://docs.litellm.ai/docs/providers/aiml))
- [Semantic Routers](https://awesome-repositories.com/f/artificial-intelligence-ml/semantic-routers.md) — Routes incoming requests to specialized models by matching input content against examples using embedding-based similarity. ([source](https://docs.litellm.ai/docs/proxy/auto_routing))
- [Batch Processors](https://awesome-repositories.com/f/artificial-intelligence-ml/batch-processors.md) — Executes batch processing tasks on supported providers via standard compatible API interfaces. ([source](https://docs.litellm.ai/docs/providers/azure))
- [Prompt Caches](https://awesome-repositories.com/f/artificial-intelligence-ml/prompt-caches.md) — Stores large context or tool definitions to reduce token usage and improve performance. ([source](https://docs.litellm.ai/docs/providers/anthropic))
- [Structured Output Converters](https://awesome-repositories.com/f/artificial-intelligence-ml/structured-output-converters.md) — Transforms schema formats into provider-specific requirements to ensure structured data returns. ([source](https://docs.litellm.ai/docs/providers/anthropic))
- [Agent Registrations](https://awesome-repositories.com/f/artificial-intelligence-ml/agent-registrations.md) — Adds various agent types into a centralized gateway to enable unified management. ([source](https://docs.litellm.ai/docs/a2a))
- [Document Rerankers](https://awesome-repositories.com/f/artificial-intelligence-ml/document-rerankers.md) — Reorders lists of documents based on specific queries using provider reranking models. ([source](https://docs.litellm.ai/docs/providers/cohere))
- [Embedding Generators](https://awesome-repositories.com/f/artificial-intelligence-ml/embedding-generators.md) — Creates numerical representations for input strings to optimize retrieval performance. ([source](https://docs.litellm.ai/docs/providers/cohere))
- [Proxy Routers](https://awesome-repositories.com/f/artificial-intelligence-ml/proxy-routers.md) — Routes requests to specific model providers by defining model mappings and API credentials. ([source](https://docs.litellm.ai/docs/providers/abliteration))

### DevOps & Infrastructure

- [Traffic Orchestrators](https://awesome-repositories.com/f/devops-infrastructure/traffic-orchestrators.md) — Routes incoming requests through a centralized gateway that manages authentication and load balancing.
- [Server Management Systems](https://awesome-repositories.com/f/devops-infrastructure/server-management-systems.md) — Configures server connections using multiple transports while enforcing access control. ([source](https://docs.litellm.ai/docs/mcp))
- [Load Balancers](https://awesome-repositories.com/f/devops-infrastructure/load-balancers.md) — Distributes traffic across multiple model deployments while monitoring rate limits and token usage. ([source](https://docs.litellm.ai/docs/providers/azure))
- [Traffic Load Balancers](https://awesome-repositories.com/f/devops-infrastructure/traffic-load-balancers.md) — Distributes traffic across multiple model deployments using routing rules and automatic fallbacks for high availability. ([source](https://docs.litellm.ai/docs/simple_proxy))

### Security & Cryptography

- [Model Safety Filters](https://awesome-repositories.com/f/security-cryptography/model-safety-filters.md) — Implements safety filters to validate or block model inputs and outputs based on custom content policies. ([source](https://docs.litellm.ai/docs/simple_proxy))
- [Content Guardrails](https://awesome-repositories.com/f/security-cryptography/content-guardrails.md) — Intercepts request and response streams to apply content moderation and safety policy enforcement.
- [Security Proxies](https://awesome-repositories.com/f/security-cryptography/security-proxies.md) — Secures model interactions with centralized authentication, content guardrails, and audit logging.
- [Access Control Systems](https://awesome-repositories.com/f/security-cryptography/access-control-systems.md) — Secures administrative access using single sign-on, audit logging, and content guardrails. ([source](https://docs.litellm.ai/docs/learn/enterprise_quickstart))
- [Proxy Authentication](https://awesome-repositories.com/f/security-cryptography/proxy-authentication.md) — Secures proxy access using API keys and identity providers to manage user permissions and enforce access control. ([source](https://docs.litellm.ai/docs/simple_proxy))
- [Access Control Policies](https://awesome-repositories.com/f/security-cryptography/access-control-policies.md) — Enforces fine-grained access control and usage tracking by passing user identifiers in request headers. ([source](https://docs.litellm.ai/docs/mcp))
- [API Key Management](https://awesome-repositories.com/f/security-cryptography/api-key-management.md) — Maintains stateful access control and usage tracking by mapping virtual API keys to specific budgets.
- [Provider Authenticators](https://awesome-repositories.com/f/security-cryptography/provider-authenticators.md) — Verifies requests to model providers by configuring required API keys. ([source](https://docs.litellm.ai/docs/providers/bedrock))
- [Proxy Security Tools](https://awesome-repositories.com/f/security-cryptography/proxy-security-tools.md) — Protects infrastructure using single sign-on, role-based access control, and secret management integrations. ([source](https://docs.litellm.ai/docs/enterprise))
- [Data Redaction Tools](https://awesome-repositories.com/f/security-cryptography/data-redaction-tools.md) — Removes sensitive information like API keys from model requests automatically using secret detection guardrails. ([source](https://docs.litellm.ai/docs/enterprise))
- [OAuth Configurations](https://awesome-repositories.com/f/security-cryptography/oauth-configurations.md) — Sets up OAuth 2.0 for servers by using automatic provider discovery or explicit credentials. ([source](https://docs.litellm.ai/docs/mcp))
- [Virtual Key Managers](https://awesome-repositories.com/f/security-cryptography/virtual-key-managers.md) — Generates virtual API keys with specific rate limits and usage tracking for managing user access. ([source](https://docs.litellm.ai/docs/proxy/docker_quick_start))

### Programming Languages & Runtimes

- [Integration SDKs](https://awesome-repositories.com/f/programming-languages-runtimes/integration-sdks.md) — Provides a unified interface for interacting with diverse language models and external tool-calling services.

### Software Engineering & Architecture

- [Complexity-Based Routers](https://awesome-repositories.com/f/software-engineering-architecture/complexity-based-routers.md) — Classifies request complexity to send queries to appropriate models based on cost and performance. ([source](https://docs.litellm.ai/docs/proxy/auto_routing))
- [Routing Configurations](https://awesome-repositories.com/f/software-engineering-architecture/routing-configurations.md) — Defines traffic distribution and model mapping rules through externalized configuration files.
- [Model Routing Configurations](https://awesome-repositories.com/f/software-engineering-architecture/model-routing-configurations.md) — Defines available models and provider-specific parameters within a configuration file. ([source](https://docs.litellm.ai/docs/proxy/docker_quick_start))

### Business & Productivity Software

- [Budget Management Systems](https://awesome-repositories.com/f/business-productivity-software/budget-management-systems.md) — Manages multi-tenant environments by enforcing budgets and spending limits across teams and projects. ([source](https://docs.litellm.ai/docs/enterprise))
- [Spend Tracking Tools](https://awesome-repositories.com/f/business-productivity-software/spend-tracking-tools.md) — Reports model usage costs by grouping requests with custom tags for granular spend analysis. ([source](https://docs.litellm.ai/docs/enterprise))

### Data & Databases

- [Response Caching](https://awesome-repositories.com/f/data-databases/response-caching.md) — Stores and retrieves previous model outputs in a cache to reduce latency and operational costs. ([source](https://docs.litellm.ai/docs/simple_proxy))

### System Administration & Monitoring

- [Usage Limiters](https://awesome-repositories.com/f/system-administration-monitoring/usage-limiters.md) — Enforces spending budgets and rate limits per user or key to control costs and prevent service abuse. ([source](https://docs.litellm.ai/docs/simple_proxy))
- [Interaction Logs](https://awesome-repositories.com/f/system-administration-monitoring/interaction-logs.md) — Monitors agent interactions by viewing request content, user metadata, and performance metrics. ([source](https://docs.litellm.ai/docs/a2a))
- [Observability Platforms](https://awesome-repositories.com/f/system-administration-monitoring/observability-platforms.md) — Captures and monitors request metadata, latency, and costs across all model interactions.
- [Activity Monitors](https://awesome-repositories.com/f/system-administration-monitoring/activity-monitors.md) — Observes proxy activity with team-based logging and durable log exports to ensure compliance. ([source](https://docs.litellm.ai/docs/enterprise))
- [Observability Callbacks](https://awesome-repositories.com/f/system-administration-monitoring/observability-callbacks.md) — Triggers asynchronous hooks to log request metadata and performance metrics to external services. ([source](https://docs.litellm.ai/observability/callbacks))
- [Request Logs](https://awesome-repositories.com/f/system-administration-monitoring/request-logs.md) — Captures and stores model request and response data for monitoring and auditing purposes. ([source](https://docs.litellm.ai/observability/supabase_integration))

### Content Management & Publishing

- [Content Moderation Services](https://awesome-repositories.com/f/content-management-publishing/content-moderation-services.md) — Filters model content using integrated moderation services with configurable thresholds. ([source](https://docs.litellm.ai/docs/enterprise))