# coaidev/coai

**Attribution required: if you use, quote, or summarise this content, you must credit and link back to [awesome-repositories.com](https://awesome-repositories.com/repository/coaidev-coai).**

9,212 stars · 1,218 forks · TypeScript · Apache-2.0

## Links

- GitHub: https://github.com/coaidev/coai
- Homepage: https://coai.dev
- awesome-repositories: https://awesome-repositories.com/repository/coaidev-coai.md

## Topics

`ai-gateway` `api` `chat` `chatgpt` `cross-platform` `gemini` `golang` `llm-gateway` `openai` `proxy` `react`

## Description

CoAI is an enterprise-grade, self-hostable AI gateway platform that unifies access to over 200 AI models from more than 35 providers through a single OpenAI-compatible API endpoint. It functions as a multi-tenant gateway, routing requests across providers with load balancing, automatic failover, and priority-based routing, while exposing standard OpenAI API endpoints for chat, image generation, model listing, and billing to enable seamless integration with existing tools and clients.

The platform distinguishes itself through a comprehensive set of operational capabilities built around the gateway. It includes a content moderation engine that scans AI-generated and user-provided content against custom safety policies, a multi-tenant token metering and billing system supporting subscription and pay-as-you-go plans, and a response caching layer that reduces latency and API costs for repeated requests. CoAI also provides a file parsing pipeline that extracts text from PDF, DOCX, PPTX, Excel, and images using OCR, a plugin-based extension system for adding new capabilities, and workflow automation for chaining AI calls into repeatable sequences.

Beyond the core gateway, CoAI offers a conversational AI assistant with knowledge base integration, web search, image generation via DALL-E, Midjourney, or Stable Diffusion, and speech recognition. It supports single sign-on authentication through SAML or OAuth, role-based access control, and real-time usage monitoring with interactive dashboards. The platform synchronizes chat history and settings across devices, and can be deployed via Docker, Kubernetes, or one-click cloud setups with elastic scaling for high availability.

## Tags

### Software Engineering & Architecture

- [Multi-Tenant AI Gateways](https://awesome-repositories.com/f/software-engineering-architecture/api-gateways/ai-provider-gateways/multi-tenant-ai-gateways.md) — Routes requests to 200+ AI models from 35+ providers with load balancing, failover, and multi-tenant billing.
- [AI Provider Gateways](https://awesome-repositories.com/f/software-engineering-architecture/api-gateways/ai-provider-gateways.md) — Routes requests to 200+ models from 35+ providers through a single OpenAI-compatible API endpoint. ([source](https://chatnio.com/))
- [Plugin Marketplaces](https://awesome-repositories.com/f/software-engineering-architecture/integration-extensibility/extensibility/third-party-plugins/plugin-marketplaces.md) — Adds new capabilities to the AI assistant through a marketplace of third-party plugins. ([source](https://chatnio.com/))

### Artificial Intelligence & ML

- [OpenAI-Compatible APIs](https://awesome-repositories.com/f/artificial-intelligence-ml/agentic-systems-frameworks/model-integration-serving/model-integration-interfaces/ai-integration-apis/openai-compatible-apis.md) — Exposes standard OpenAI API endpoints for chat, image generation, model listing, and billing. ([source](https://cdn.jsdelivr.net/gh/coaidev/coai@main/README.md))
- [AI Chat Assistants](https://awesome-repositories.com/f/artificial-intelligence-ml/ai-chat-assistants.md) — Interacts with a conversational AI that supports file analysis, web search, image generation, and speech recognition. ([source](https://chatnio.com/))
- [Model Request Routing](https://awesome-repositories.com/f/artificial-intelligence-ml/ai-model-clients/model-request-routing.md) — Routes user requests to any of 200+ large language models from 35+ providers through a single unified API gateway. ([source](https://cdn.jsdelivr.net/gh/coaidev/coai@main/README.md))
- [AI Model Integrations](https://awesome-repositories.com/f/artificial-intelligence-ml/ai-model-integrations.md) — Connects private, self-hosted, and third-party AI models through a single OpenAI-compatible API.
- [Private Model Connectors](https://awesome-repositories.com/f/artificial-intelligence-ml/ai-provider-integrations/private-model-connectors.md) — Provides connectors for integrating self-hosted or custom AI models alongside third-party providers. ([source](https://chatnio.com/))
- [AI Service Configurations](https://awesome-repositories.com/f/artificial-intelligence-ml/ai-service-configurations.md) — Connects and manages external AI providers by setting up channels with model-specific configurations. ([source](https://chatnio.com/docs))
- [OpenAI-Compatible](https://awesome-repositories.com/f/artificial-intelligence-ml/artificial-intelligence-tooling/agent-and-tool-integrations/api-servers/openai-compatible.md) — Exposes standard OpenAI API endpoints for chat, image generation, model listing, and billing.
- [Multi-Provider Abstractions](https://awesome-repositories.com/f/artificial-intelligence-ml/model-provider-integrations/multi-provider-abstractions.md) — Routes requests to 200+ models from 35+ providers through a single integration point. ([source](https://chatnio.com/docs))
- [Self-Hosted AI Platforms](https://awesome-repositories.com/f/artificial-intelligence-ml/self-hosted-ai-platforms.md) — Runs the entire multi-tenant AI gateway on private infrastructure using Docker or Kubernetes.
- [Third-Party Model Integration](https://awesome-repositories.com/f/artificial-intelligence-ml/third-party-model-integration.md) — Connects self-hosted or custom AI models alongside third-party providers for unified access. ([source](https://chatnio.com/))
- [Automation Workflows](https://awesome-repositories.com/f/artificial-intelligence-ml/automation-workflows.md) — Chains AI calls and other actions into repeatable sequences to streamline repetitive tasks. ([source](https://chatnio.com/))
- [Custom AI Models](https://awesome-repositories.com/f/artificial-intelligence-ml/custom-ai-models.md) — Adds private or proprietary models alongside the built-in provider catalog for bespoke use cases. ([source](https://chatnio.com/docs))
- [Custom Model Service Integrations](https://awesome-repositories.com/f/artificial-intelligence-ml/custom-model-service-integrations.md) — Adds proprietary or self-hosted AI models alongside the built-in provider catalog. ([source](https://chatnio.com/docs))
- [Text-to-Image Generators](https://awesome-repositories.com/f/artificial-intelligence-ml/generative-ai-resources/diffusion-visual-models/generative-ai-pipelines/text-to-image-generators.md) — Creates images using DALL-E, Midjourney, or Stable Diffusion through a unified text-to-image interface. ([source](https://cdn.jsdelivr.net/gh/coaidev/coai@main/README.md))
- [Real-Time Speech Transcribers](https://awesome-repositories.com/f/artificial-intelligence-ml/speech-to-text-libraries/real-time-speech-transcribers.md) — Transcribes spoken input into text for voice-driven interaction with the assistant. ([source](https://chatnio.com/))
- [Web Search Integrations](https://awesome-repositories.com/f/artificial-intelligence-ml/web-search-integrations.md) — Performs internet searches using engines like Google, Bing, and Wikipedia, then feeds results into the selected AI model. ([source](https://cdn.jsdelivr.net/gh/coaidev/coai@main/README.md))

### Business & Productivity Software

- [Billing and Usage](https://awesome-repositories.com/f/business-productivity-software/financial-operational-management/billing-financial-systems/billing-and-usage.md) — Logs all API calls and charges users based on consumption with subscription or pay-as-you-go billing. ([source](https://chatnio.com/))
- [Conversational Knowledge Base Assistants](https://awesome-repositories.com/f/business-productivity-software/knowledge-content-creation/knowledge-information-management/knowledge-management-platforms/ai-integrated-knowledge-bases/conversational-knowledge-base-assistants.md) — Answers questions, analyzes files, and generates images through a conversational interface that integrates domain-specific documents and web search.
- [Usage-Based Billing Integrations](https://awesome-repositories.com/f/business-productivity-software/multi-tenant-billing-infrastructures/usage-based-billing-integrations.md) — Controls user access with role-based permissions and tracks usage for subscription or pay-as-you-go billing. ([source](https://chatnio.com/))
- [User and Subscription Administration](https://awesome-repositories.com/f/business-productivity-software/billing-and-subscription-management/user-and-subscription-administration.md) — Provides a backend dashboard for managing users, subscriptions, gift codes, announcements, pricing, and site branding. ([source](https://cdn.jsdelivr.net/gh/coaidev/coai@main/README.md))
- [Multi-Tenant Billing Infrastructures](https://awesome-repositories.com/f/business-productivity-software/multi-tenant-billing-infrastructures.md) — Measures and bills API consumption per tenant using token-based metering with subscription and pay-as-you-go plans.
- [Usage-Based Billings](https://awesome-repositories.com/f/business-productivity-software/usage-based-billings.md) — Charges for AI service consumption using token-based and subscription billing models. ([source](https://chatnio.com/docs))
- [Usage Metering & Billing](https://awesome-repositories.com/f/business-productivity-software/usage-metering-billing.md) — Tracks and charges for AI API consumption using token metering, subscriptions, and pay-as-you-go plans.

### DevOps & Infrastructure

- [LLM](https://awesome-repositories.com/f/devops-infrastructure/api-service-management/api-management/api-documentation-publishing/api-management-platforms/llm.md) — Manages, monitors, and bills multi-model AI API access with built-in caching, moderation, and usage analytics.
- [Self-Hosted AI Platforms](https://awesome-repositories.com/f/devops-infrastructure/self-hosted-ai-platforms.md) — Unifies private and third-party AI models with web search, file parsing, and workflow automation.
- [Self-Hosted Deployment Platforms](https://awesome-repositories.com/f/devops-infrastructure/self-hosted-deployment-platforms.md) — Provides full source code access for running the entire system on private infrastructure. ([source](https://chatnio.com/))
- [AI Model Load Balancers](https://awesome-repositories.com/f/devops-infrastructure/traffic-load-balancers/ai-model-load-balancers.md) — Distributes traffic among AI providers based on availability, cost, and performance to optimize reliability and expense. ([source](https://chatnio.com/))
- [One-Click Deployments](https://awesome-repositories.com/f/devops-infrastructure/cloud-agent-orchestration/cloud-agent-deployers/one-click-deployments.md) — Launches a full instance using Docker Compose, Zeabur, or Kubernetes for rapid setup. ([source](https://chatnio.com/docs))
- [Application Cloud Deployments](https://awesome-repositories.com/f/devops-infrastructure/cloud-deployment/multi-cloud-deployments/cross-cloud-certificate-deployment/cloud-platform-deployments/application-cloud-deployments.md) — Launches the platform on cloud services with automated scaling and management. ([source](https://chatnio.com/))
- [Elastic Scaling Deployments](https://awesome-repositories.com/f/devops-infrastructure/container-cluster-deployments/elastic-scaling-deployments.md) — Spans a cluster of servers that automatically grows or shrinks capacity to maintain high availability using Docker and Kubernetes.
- [Cloud or Self-Hosted Deployments](https://awesome-repositories.com/f/devops-infrastructure/feature-flags/cloud-or-self-hosted-deployments.md) — Runs on Docker, Kubernetes, or cloud SaaS platforms with automated scaling and minimal resource overhead. ([source](https://chatnio.com/))
- [High Availability Deployments](https://awesome-repositories.com/f/devops-infrastructure/high-availability-deployments.md) — Supports multi-node production deployments with redundancy patterns for high availability. ([source](https://chatnio.com/docs))

### Security & Cryptography

- [API Token Distributions](https://awesome-repositories.com/f/security-cryptography/access-restrictions/multi-tenant/api-token-distributions.md) — Distributes API tokens and controls access for multiple users or teams within a single platform. ([source](https://chatnio.com/))
- [Permission-Based Access Control](https://awesome-repositories.com/f/security-cryptography/permission-based-access-control.md) — Manages user accounts and roles to restrict access to AI services and administrative functions. ([source](https://chatnio.com/docs))
- [Role-Based Access Control](https://awesome-repositories.com/f/security-cryptography/role-based-access-control.md) — Assigns permissions to users based on their role to restrict access to AI services and administrative functions. ([source](https://chatnio.com/))
- [Content Moderation](https://awesome-repositories.com/f/security-cryptography/content-moderation.md) — Scans AI-generated content against safety policies to enforce enterprise compliance rules. ([source](https://chatnio.com/docs))
- [Content Moderation Filters](https://awesome-repositories.com/f/security-cryptography/content-moderation-filters.md) — Scans AI-generated and user-provided content against custom safety policies to enforce compliance.
- [Enterprise SSO Authentication](https://awesome-repositories.com/f/security-cryptography/oauth-authentication/enterprise-sso-security/enterprise-sso-authentication.md) — Logs in using existing identity providers like SAML or OAuth for single sign-on access. ([source](https://chatnio.com/))
- [Content Moderation Filters](https://awesome-repositories.com/f/security-cryptography/security-and-access-control/content-moderation-filters.md) — Scans text against custom rules and patterns to filter inappropriate or policy-violating content. ([source](https://chatnio.com/))

### Web Development

- [AI Provider Routing](https://awesome-repositories.com/f/web-development/api-endpoint-configurations/service-endpoint-configurations/ai-provider-routing.md) — Distributes API calls among multiple LLM providers using load balancing, failover, and priority-based routing. ([source](https://chatnio.com/docs))
- [OpenAI-Compatible Servers](https://awesome-repositories.com/f/web-development/openai-compatible-servers.md) — Exposes standard OpenAI API endpoints for chat, image generation, and model listing.
- [Provider-Agnostic LLM Routing](https://awesome-repositories.com/f/web-development/provider-agnostic-llm-routing.md) — Routes API requests across multiple providers using priority-based load balancing and automatic failover for high availability.
- [Docker Deployments](https://awesome-repositories.com/f/web-development/self-hosted-file-servers/docker-deployments.md) — Packages the platform into containers for consistent setup across development and production environments. ([source](https://chatnio.com/))

### Part of an Awesome List

- [Web Search Answer Retrievers](https://awesome-repositories.com/f/awesome-lists/ai/question-answering/web-search-answer-retrievers.md) — Retrieves live information from the internet to answer queries through the assistant. ([source](https://chatnio.com/))

### Data & Databases

- [Document Format Parsers](https://awesome-repositories.com/f/data-databases/binary-and-text-file-parsing/document-format-parsers.md) — Extracts text from PDF, DOCX, PPTX, Excel, and images using OCR, and stores files on cloud storage. ([source](https://cdn.jsdelivr.net/gh/coaidev/coai@main/README.md))
- [Multi-Format Document Parsers](https://awesome-repositories.com/f/data-databases/data-engineering-infrastructure/data-extraction-ingestion/data-ingestion/document-parsing-pipelines/excel-file-parsing-pipelines/multi-format-document-parsers.md) — Extracts text from PDF, DOCX, PPTX, Excel, and images using OCR, and stores files on cloud storage.
- [Auto-Scaling Clusters](https://awesome-repositories.com/f/data-databases/horizontal-scaling/elastic-kubernetes-scaling/auto-scaling-clusters.md) — Spans a cluster of servers that automatically grows or shrinks capacity to maintain high availability. ([source](https://chatnio.com/))
- [Knowledge Base Integrations](https://awesome-repositories.com/f/data-databases/knowledge-base-integrations.md) — Feeds domain-specific documents into the assistant so it answers with accurate, private information. ([source](https://chatnio.com/))
- [Response Caching](https://awesome-repositories.com/f/data-databases/response-caching.md) — Caches identical model outputs to serve repeated requests instantly, reducing latency and API costs.
- [Multi-Model AI Billing Systems](https://awesome-repositories.com/f/data-databases/usage-billing-systems/multi-model-ai-billing-systems.md) — Tracks token-based and subscription usage across multiple AI providers with credit and pay-as-you-go plans.

### System Administration & Monitoring

- [Real-Time Monitoring Dashboards](https://awesome-repositories.com/f/system-administration-monitoring/real-time-monitoring-dashboards.md) — Tracks API usage and performance metrics with live dashboards and comprehensive statistics. ([source](https://chatnio.com/docs))
- [AI Usage and Routing Dashboards](https://awesome-repositories.com/f/system-administration-monitoring/usage-monitoring/usage-dashboard-builders/ai-usage-and-routing-dashboards.md) — Displays real-time metrics, cost data, and performance analytics with interactive visualizations. ([source](https://chatnio.com/))
- [AI Usage Dashboards](https://awesome-repositories.com/f/system-administration-monitoring/web-based-dashboards/ai-usage-dashboards.md) — Visualizes real-time metrics, spending, and performance data in interactive charts with drill-down capabilities. ([source](https://chatnio.com/))