New Api

This project is an AI model API gateway and proxy server designed to provide a unified interface for interacting with diverse artificial intelligence service providers. It functions as a centralized middleware platform that routes, load balances, and translates API requests across multiple models, enabling developers to access text, image, audio, and video generation capabilities through a single, standardized integration.

The gateway distinguishes itself through comprehensive administrative and financial controls, including event-driven usage accounting, real-time token consumption tracking, and granular role-based access control. It supports complex traffic management by distributing requests across multiple credential pools and providers to optimize throughput and bypass rate limits. Furthermore, it integrates a robust identity federation system that supports OIDC, OAuth, and hardware-backed passkeys to secure user access and manage multi-tenant environments.

Beyond core routing, the platform provides extensive tooling for service maintenance, including automated health checks, model registry synchronization, and content moderation filters. It also features a complete billing and payment infrastructure, allowing administrators to manage user credit balances, process prepaid redemptions, and monitor cost structures across different model vendors.

The system is designed for flexible deployment across containerized and distributed infrastructure, with administrative interfaces for auditing usage logs, managing API channels, and configuring global system parameters.

Features

Model Gateways - Acts as a centralized gateway that routes, load balances, and translates requests across multiple AI model providers.
Chat Completion Services - Produces model responses based on chat history while maintaining compatibility with standard industry chat completion interfaces.
Image Generation - Creates visual content from text descriptions by routing prompts to configured image generation models.
Model API Gateways - Provides a unified API interface for invoking diverse text, image, audio, and video generation models.
Model Proxy Gateways - Provides a unified interface for routing and translating requests across multiple LLM providers.
Billing and Usage - Monitors request logs, calculates costs, and enforces spending limits across diverse model endpoints.
API Usage Proxies - Standardizes API interactions and manages usage quotas for generative AI services as a middleware proxy.
Model Request Proxies - Intercepts and routes API requests to interchangeable language model providers with streaming support.
Access Token Management - Generates secure credentials for programmatic access to gateway services.
Bearer Token Authentication - Secures API requests and management interfaces using bearer token and session-based authentication.
System Configuration Management - Provides centralized management for backend settings, API channels, and model availability.
Text-to-Video Generators - Synthesizes dynamic video content from descriptive text prompts using generative models.
Services - Translates requests into video generation tasks for compatible models through a unified interface.
Usage Metering & Billing - Tracks token consumption and costs per request to manage quotas and billing.
AI Traffic Routing - Distributes incoming API traffic across multiple providers using weighted load balancing and retries.
API Access Security - Validates incoming API requests against centralized credentials to ensure authorized access to model endpoints.
OpenID Connect Providers - Secures API access using OpenID Connect, OAuth, and hardware-backed passkeys for multi-user management.
Role-Based Access Control - Enforces hierarchical access control by assigning users to specific privilege levels.
API Translation Layers - Adapts proprietary model formats to standard industry API specifications for unified routing.
Audio Transcription - Converts spoken audio content into written text through automated transcription services.
Credential Retrievers - The AI Gateway fetches secret keys associated with model provider channels to enable administrative auditing or manual configuration.
Model Provider Management - Provides administrative tools to audit and oversee available service providers and their metadata.
Text Generation APIs - Processes prompts and returns text completions formatted to match standard industry API specifications.
Text-to-Audio Synthesis - Synthesizes speech from text input using integrated language models.
Usage Quotas - Tracks consumption statistics and remaining allowances to enforce individual service limits.
Video Generation - Submits requests to create video content and retrieves task results through a centralized interface.
Payment Processing - Collects and processes user payments to maintain account balances for AI service consumption.
Store Credit Systems - Enables users to redeem prepaid codes to increase their account credit balances.
Usage Accounting - Tracks token consumption and billing metrics in real-time to enforce spending limits.
Load Balancers - Distributes traffic across service channels and credential pools to optimize throughput.
Redemption Code Generators - The AI Gateway creates unique alphanumeric tokens that users can redeem to add balance or credits to their accounts.
Content Moderation - Analyzes input and output content to ensure compliance with safety and moderation policies.
Request Access Restrictions - Enforces credential-based access restrictions on incoming requests to ensure only authorized entities interact with services.
Usage Quotas - Manages usage quotas and spending limits for specific service channels to control consumption.
Identity Federation Providers - Integrates external identity providers and hardware-backed credentials for secure multi-tenant access.
OIDC Authentication Plugins - Delegates login processes to external OpenID Connect providers and handles callbacks to establish secure sessions.
Policy-Based Access Control - Enforces security permissions through token grouping and model-level access restrictions.
Model Provider Integrations - Fetches a list of all models currently configured and accessible across connected service providers.
Model Registries - Retrieves a comprehensive list of supported models for seamless client application integration.
Image Editing - Modifies existing images using text prompts via generative inpainting and transformation models.
Response Translation Layers - Converts raw model outputs into standardized API response structures based on client headers.
Billing Management - Provides administrative tools to manually adjust user credit balances for billing purposes.
Connection Channel Management - Registers and manages external model providers by defining credentials, base URLs, and model availability.
Server Metadata Registries - Maintains a unified database of model configurations and credentials for consistent routing.
Redemption Code Modifiers - The AI Gateway updates existing redemption codes by changing associated balances, usage limits, or expiration settings.
Revocations - Enables immediate invalidation of authentication tokens to terminate access to gateway services.
API Key Management - Manages pools of API credentials to distribute traffic and bypass provider rate limits.
Cross-Site Request Forgery Protections - Generates secure identifiers for OAuth flows to protect against cross-site request forgery.
Channel Restrictions - Limits availability of specific model providers to authorized users by validating security tags on channels.
Identity Authentication - Connects multiple identity providers to manage user access and secure entry points for gateway services.
Redemption Code Detail Accessors - The AI Gateway fetches specific configuration and status of single redemption codes by identifier for administrative review.
User Group Management - Configures permissions and access levels for user groups to control resource interaction.
AI Connection Diagnostics - Verifies service reachability through automated health checks against provider endpoints.
Document Rerankers - Improves information retrieval accuracy by sorting document collections based on relevance to specific queries.
Image Editing Workflows - Provides workflows for generating modified or extended versions of existing images via text instructions.
Model Metadata Registrars - Defines configuration details for external models to enable centralized routing and cost accounting.
Model Provider Configurations - Manages external machine learning model providers, credentials, and status for centralized integration.
Realtime AI Session Managers - Facilitates persistent, bidirectional communication sessions for interactive AI applications.
Usage Analytics - Provides visual dashboards for analyzing request patterns and model performance.
Video Generation Services - Fetches binary streams of completed video generation tasks from model providers.
Usage Billing Systems - Fetches current pricing and cost structures for models to support accurate budget tracking.
API Usage Analytics - Audits historical request and response logs to analyze system activity and performance.
Group Membership Enforcement - Validates user access levels by fetching authorization groups assigned to authenticated users.
Multi-Factor Authentication Strategies - Enforces secondary verification requirements during login to strengthen account security.
GitHub OAuth Providers - Redirects users to GitHub for authorization and exchanges codes for access tokens to establish secure sessions.
Discord Identity Providers - Verifies user identity through Discord OAuth to manage access and session state.
Asynchronous Task Queues - Manages the lifecycle of long-running operations like video generation through background task queues.
Group Removals - The AI Gateway removes prefill groups from the system, requiring elevated administrator permissions to ensure authorized modification.
Response Streaming Interfaces - Streams generated model responses to clients in real-time to minimize latency.
Model Configuration - Modifies settings and metadata for available models to control availability, pricing, and routing.
Model Registries - Maintains a registry of available models to ensure local configurations reflect upstream capabilities.
Multimodal Processing - Transmits encoded media files like images and PDFs to models for multimodal analysis.
Vector Embeddings - Transforms input text into numerical vector representations for semantic search and analysis.
Billing and Subscription Management - Synchronizes account status and balances by processing incoming payment notifications from external providers.
Resource Usage Monitoring - Fetches quota and consumption data to monitor resource utilization across models.
Payment Gateways - Retrieves payment status and credit balances from integrated billing providers.
Usage Restrictions - Prevents specific API channels from being used by disabling them based on assigned tags.
Balance Synchronizers - Updates remaining quotas and credit balances for configured service channels to ensure accurate tracking.
Cleanup Operations - Maintains system security by automatically removing unused or expired access tokens.
Modifications - Allows administrators to update token metadata and permissions to adjust scope or lifecycle.
Redemption Code Revocations - Prevents further use of resources by invalidating specific redemption codes or credit vouchers.
Community Platform Providers - Verifies user identity by allowing sign-in through existing account credentials from community platforms.
Passkey Authentication - Requests cryptographic challenges from the server to verify user identity through hardware-backed credentials.
User Access Revocation - Invalidates active authentication tokens to terminate user sessions and revoke access.
User Account Management - Manages the creation of new user accounts and the assignment of initial permissions.
Health Checks - Verifies whether the gateway is fully configured and ready to process requests via operational status checks.
Task Monitoring - Lists active and completed system tasks to allow administrators to monitor asynchronous operation progress.

vercel/vercel

15,738View on GitHub

Vercel is a cloud platform for building, deploying, and scaling web applications. It provides a unified infrastructure that automates the build process by detecting project frameworks and distributing static and dynamic content through a global content delivery network. The platform executes application logic using serverless functions that scale automatically based on real-time traffic demand. The platform distinguishes itself through a centralized AI gateway that proxies requests to multiple model providers, enabling standardized authentication, observability, and cost tracking. It supports

alibaba/higress

7,558View on GitHub

Higress is an AI API gateway and cloud-native traffic manager that functions as a Kubernetes ingress controller. It provides a centralized system for routing, securing, and optimizing traffic directed toward large language models, AI agents, and microservice architectures. The project distinguishes itself through deep AI orchestration, including the ability to host and manage Model Context Protocol servers that transform REST APIs into tools for AI agents. It features specialized AI infrastructure for model request proxying, protocol translation across multiple providers, and semantic-based c

quarkusio/quarkus

15,479View on GitHub

Quarkus is a Kubernetes-native Java framework designed for building high-performance, memory-efficient applications. It utilizes ahead-of-time native compilation to transform Java code into standalone, optimized binaries that eliminate the need for a virtual machine, enabling rapid startup and reduced memory consumption. By performing code augmentation during the build phase, it shifts heavy processing tasks away from runtime, ensuring that applications are optimized for cloud-native environments. The framework distinguishes itself through a unified approach to reactive and imperative program

openai/openai-go

2,974View on GitHub

openai-go is an LLM SDK for Go and a client for interacting with OpenAI services. It provides type-safe bindings to generate text, images, and audio via REST endpoints, enabling the integration of large language models and AI assistant orchestration into Go applications. The library serves as an agent orchestration tool for managing stateful conversation threads and autonomous agents with integrated tool calling and file search. It also functions as an asynchronous batch processing client for monitoring large-scale request groups and fine-tuning jobs, alongside a management SDK for controllin

QuantumNousnew-api

Features