Awesome GitHub RepositoriesServer-Side REST API Frameworks

ds4 is a local inference engine for DeepSeek models that includes a distributed runtime for splitting transformer layers across networked computers. It functions as a reasoning controller with a local weight streamer and an API server that streams chat completions via industry standard endpoints. The system employs a memory management model that loads model experts from disk on demand to execute models that exceed available system RAM. It provides controls for reasoning effort and model behavior steering, allowing the modification of response characteristics through activation directions. Th

This project is a full-stack project management web application designed for organizing work and managing project workflows. It consists of a Node.js TypeScript REST API for server-side business logic and a React component UI library for the frontend interface. The application implements a board-based task visualization system with customizable status workflows. It provides mechanisms for issue workflow management, including task prioritization and team member assignment to track responsibilities and progress. The system is built with a decoupled client-server architecture using full-stack T

WildfireChat’s im-server is an open-source instant messaging backend that handles message routing, user management, and group chat for private deployments. It provides a self-hosted messaging platform that can run on private infrastructure without third-party cloud dependencies, supporting enterprise messaging with group management, user directories, and external system integration. The server synchronizes messages and state across mobile, desktop, web, and mini-program clients simultaneously, and supports real-time audio/video calls with group conferencing and live streaming. It includes enc

gpt4free-ts is a TypeScript-based LLM API proxy and gateway that provides a unified interface for accessing large language models without paid subscriptions or official API keys. It functions as a containerized AI bridge that routes requests to various free third-party providers to retrieve chat completions. The project acts as an OpenAI API wrapper, translating requests and responses into the standard OpenAI chat completions format to ensure compatibility with existing AI tools. It utilizes a provider-based routing system to distribute request loads across available endpoints. The gateway s

Aim is an open-source platform for logging, visualizing, and comparing machine learning training runs and LLM traces. It provides a remote tracking server and a comparison UI, functioning as an ML experiment tracker, AI workflow logger, and LLM trace recorder that captures prompts, generations, and tool calls from AI applications. The platform distinguishes itself through a run-based data model with local SQLite storage, real-time metric streaming, and a plugin-based explorer system that supports specialized visual analysis of metrics, images, audio, and text. It offers a Python SDK with cont

The free AI already on your Mac. CLI tool, OpenAI-compatible server, and interactive chat — all on-device via Apple Intelligence. No API keys, no cloud, no downloads.

LiteRT-LM is a high-performance inference framework designed to execute large language models locally on mobile, desktop, and IoT hardware. It serves as an on-device model runtime that utilizes CPU, GPU, and NPU acceleration to provide low-latency processing. The framework is distinguished by its ability to process text, vision, and audio inputs through a single multi-modal inference engine. It features a local HTTP server that emulates OpenAI-compatible API endpoints and a WebGPU-based runtime for executing models directly within a web browser. To ensure output reliability, it includes a con

KServe is a Kubernetes-native platform for deploying and serving machine learning models as scalable inference services. It supports both generative AI models, including large language models, and traditional predictive models from frameworks such as TensorFlow, PyTorch, Scikit-Learn, XGBoost, and ONNX. The platform manages the full lifecycle of model deployments, including revision tracking, canary rollouts, A/B testing, and automatic rollbacks, and provides serverless scale-to-zero capabilities for cost-efficient resource management. KServe distinguishes itself through a standardized infere

LoopBack Next 是一个用于构建 REST 和多协议 API 的 Node.js API 框架。它作为一个 OpenAPI 服务器实现，既可以从代码生成机器可读的规范，也可以从现有规范生成实现控制器和模型。 该框架的特色在于中央依赖注入容器和存储库模式的数据访问层。这种架构将应用程序逻辑与组件构建和持久化存储解耦，允许通过标准化的连接器系统实现数据源和业务逻辑隔离的可插拔系统。 该项目涵盖了广泛的功能，包括具有可插拔身份验证策略的基于角色的访问控制，以及外部 REST 和 SOAP 服务的编排。它还通过命令行界面提供用于 WebSocket 端点实时通信、JSON 模式验证和自动化项目脚手架的工具。 开发过程由一套 CLI 工具支持，用于引导应用程序、生成 API 组件和管理项目依赖项。

codeigniter-restserver 是一个 REST API 框架和控制器库，用于在 CodeIgniter PHP 环境中构建 RESTful 服务器。它作为一个后端实现，处理标准 HTTP 方法，通过结构化端点暴露数据和功能。 该项目包含一个可自定义的响应引擎，允许通过自定义格式化方法将输出数据转换为各种特定格式。 该库提供了将传入的 HTTP 请求映射到控制器方法、管理资源响应以及实现基于配置的访问控制的工具。

Llama-swap 是一个用于大语言模型的本地推理编排器和 API 网关。它作为一个 OpenAI API 代理，管理多个本地模型服务器的生命周期，根据传入的请求标识符自动启动和停止服务器以切换模型。 该项目通过动态模型切换和硬件优化脱颖而出。它利用专门的矩阵式并发控制来定义哪些模型可以同时运行，并采用基于成本的驱逐策略，根据相对资源成本从内存中移除不活跃的服务器。 系统提供全面的模型管理功能，包括标识符别名、请求过滤以及容器或虚拟机的生命周期命令执行。它还包含可观测性工具，如可视化模型测试平台、实时系统性能监控，以及 API 密钥验证和 TLS 加密等安全功能。 配置更新通过动态重载处理，监控文件系统变化，无需手动重启。

Crow is a C++ web framework for building HTTP servers, providing routing, a middleware pipeline, JSON serialization, and WebSocket support. It enables developers to define typed route handlers that extract URL parameters at compile time, return structured JSON responses, and manage real-time bidirectional communication. The framework distinguishes itself with compile-time route pattern parsing and typed argument binding, which eliminate runtime parsing overhead for URL parameters. Its event loop runs on Boost.Asio, and handler execution can be offloaded to a configurable thread pool to keep I

该项目是一个大语言模型账户管理系统和多用户代理。它提供了一个网关，允许多个经过身份验证的用户共享单个高级 AI 订阅或 OpenAI API 密钥。 系统作为代理层运行，拦截客户端请求并将其转发给官方 API，同时注入共享凭据。它包括一个用于集中式凭据管理的后端，以及一个用于验证用户身份的基于令牌的访问控制系统。 为了保持隐私和组织性，该项目实现了基于会话的上下文隔离，以防止用户之间的对话泄露。它还具有用于活动审计的中间件，以跟踪使用情况并监控单个用户的行为。 该界面提供对高级模型功能的访问，包括图像生成、文件分析和代码执行。

CTranslate2 is a C++ inference engine and runtime for Transformer models, designed to execute models on both CPU and GPU with optimizations for speed and memory efficiency. It functions as a model format converter, quantization tool, and REST API server, enabling deployment of neural machine translation, automatic speech recognition, and text generation models. The engine distinguishes itself through a suite of runtime optimizations including layer fusion, weight-matrix quantization, batch-by-length grouping, and a caching allocator that reuses GPU memory. It supports tensor-parallel model di

KoboldAI-Client 是一个用于与大型语言模型交互的基于 Web 的界面和工具包。它充当用于故事创作和对话式 AI 的本地 AI 文本生成器，为托管在本地硬件或云端环境中的模型提供前端。 该系统包括一个角色管理器，使用外部模块和软提示（soft-prompting）来引导 AI 响应特定的角色和写作风格。它还提供了一个 API 包装器，暴露了一个标准化的、兼容 OpenAI 的 REST API，允许外部应用与托管模型进行通信。 该平台支持多种写作、游戏和聊天机器人交互模式，并包括沙盒脚本以自动化数据处理并过滤模型输入和输出。部署选项范围从私有本地执行到容器化云 GPU 环境。

jscamp is a full-stack web development and education project focused on mastering JavaScript, TypeScript, and AI integration. It provides a structured curriculum and interactive exercises covering language fundamentals, frontend engineering, and backend API development. The project distinguishes itself through the implementation of autonomous AI agents capable of complex task automation, such as modifying files, managing servers, and executing API calls. It includes advanced AI development tools for conversational querying, real-time code suggestions, and automated repository analysis to gene

Dev-Cpp is a comprehensive development suite that serves as a C++ integrated development environment, a cross-platform application builder, and a visual UI designer. It provides a toolchain for writing, compiling, and debugging native C++ applications on Windows, while offering a framework to create native binaries for desktop, mobile, and IoT devices from a single codebase. The project distinguishes itself by integrating an embedded SQL database engine and a REST API development platform directly into the workflow. It includes an AI-assisted coding tool that leverages large language models t

This project is a PHP-based framework designed for the automated development of standardized web services. It functions as a backend toolkit that maps internal data models to REST and GraphQL endpoints, providing built-in support for resource management, data serialization, and hypermedia-aware API operations. The framework distinguishes itself through a metadata-driven architecture that uses class-level attributes to automatically derive API schemas, validation rules, and documentation. By analyzing these definitions at runtime, it generates machine-readable specifications and interactive in