Awesome GitHub RepositoriesModel Serving Endpoints

ChatGLM2-6B is an open-weight large language model designed for natural language conversations and text generation in both English and Chinese. It functions as a bilingual chat model capable of processing and maintaining coherence across text sequences up to 32K tokens. The model is optimized for local deployment through precision quantization, which reduces memory requirements to allow execution on consumer-grade hardware. It supports distributing model weights across multiple graphics cards to handle parameters that exceed the memory of a single device. The project covers capabilities for

Ludwig is a multimodal machine learning platform and low-code framework designed for building, training, and deploying neural networks. It enables the construction of models that process text, images, audio, and tabular data through a unified interface using declarative configuration files rather than custom code. The system features a specialized low-code framework for large language models, supporting supervised fine-tuning, preference alignment, and a constrained decoding tool to force structured data output via logit extraction. It also includes an automated model architecture search to i

Promptflow este un framework de dezvoltare și un orchestrator pentru construirea de aplicații bazate pe modele de limbaj mari (LLM). Acesta funcționează ca o suită de instrumente pentru proiectarea, orchestrarea și implementarea fluxurilor de lucru AI prin conectarea prompt-urilor, a codului Python personalizat și a modelelor de limbaj în secvențe executabile. Proiectul se distinge printr-un designer vizual de fluxuri de lucru AI care permite crearea de grafuri aciclice direcționate (DAG) de noduri logice. Oferă un mediu dedicat de prompt engineering pentru versionarea și compararea șabloanelor, alături de trasarea execuției cu stare pentru a înregistra apelurile de funcții și valorile variabilelor în vederea depanării pas cu pas. Platforma acoperă o gamă largă de capabilități, inclusiv RAG (Retrieval Augmented Generation) prin căutări în baze de date vectoriale și pipeline-uri de evaluare bazate pe metrici pentru testare în loturi și asigurarea calității. Gestionează întregul ciclu de viață, de la dezvoltare la producție, prin implementare containerizată, servirea endpoint-urilor de flux de lucru și gestionarea securizată a conexiunilor pentru credențialele API. Sunt furnizate o interfață în linie de comandă (CLI) și un SDK pentru validarea fluxurilor de lucru și integrarea în pipeline-uri CI/CD automatizate.

This repository is a collection of Jupyter notebooks providing reference implementations and templates for building, training, and deploying machine learning models using Amazon SageMaker. It serves as an example library for implementing model architectures and automating the machine learning lifecycle. The library provides practical patterns for machine learning training, data engineering, and model deployment. It includes implementation guides for MLOps, including workflows for model monitoring, lineage tracking, and hyperparameter tuning. The examples cover a broad range of capabilities i

Cosmos is an open platform of world models, datasets, and tools for building physical AI systems such as robots and autonomous vehicles. It provides video generation and video understanding models that can generate synthetic videos and world simulations from text, image, video, or action inputs, and analyze videos to produce captions, event timestamps, spatial bounding boxes, and next-action predictions. The platform includes a world simulation generator that produces images, videos, synchronized audio, and action-conditioned rollouts for synthetic data, alongside a visual content analyzer th

MMSegmentation is an open-source semantic segmentation toolbox built on PyTorch that provides a modular, configurable framework for building, training, evaluating, and deploying segmentation models. At its core, it offers a config-driven pipeline that assembles training, evaluation, and inference workflows by parsing hierarchical configuration files, with a modular component registry that enables plug-and-play composition of neural network modules, optimizers, datasets, and metrics. The framework supports the full model lifecycle through a unified runner interface that controls training, testi

TensorFlow Serving is a high-performance machine learning inference server designed to deploy TensorFlow models to production environments. It functions as a complete serving system that executes predictions on input data through a graph executor, providing network endpoints that eliminate the need for a separate runtime environment for client applications. The system is distinguished by its model version manager, which organizes and selects specific model versions within a directory hierarchy. It uses a filesystem watcher to detect new model versions and trigger automatic updates without int

The AWS SDK for PHP is a software development kit that provides HTTP client classes for every supported Amazon Web Service, enabling PHP applications to send authenticated requests and receive structured, typed response objects. It includes a credential resolution chain that automatically locates credentials from environment variables, instance profiles, or configuration files, and supports promise-based asynchronous execution for running multiple API calls concurrently to improve throughput. The SDK distinguishes itself through a middleware pipeline architecture that allows interception and

PaddleX is a PaddlePaddle-based framework for building, deploying, and fine-tuning AI model pipelines, with pre-built support for computer vision, OCR, document analysis, and time series tasks. It offers a toolkit of ready-to-use pipelines for image classification, object detection, segmentation, and pose estimation, alongside an end-to-end OCR document analysis pipeline that extracts text, tables, formulas, and layout information. The platform also includes a dedicated time series forecasting pipeline for analyzing historical data to detect anomalies, classify patterns, and predict future val

FATE is an open-source federated learning platform that enables multiple organizations to collaboratively train machine learning models without exposing raw data to any party. It provides a complete framework for private data collaboration, allowing participants to jointly compute on sensitive information while maintaining data privacy and security guarantees through secure multi-party computation protocols. The platform distinguishes itself through its comprehensive infrastructure management capabilities, supporting automated deployment of multi-party clusters using Ansible-driven provisioni

ToolBench is an open platform for training, serving, and evaluating large language models that retrieve and call real-world APIs to complete user instructions. It provides an API-aware inference engine that selects relevant tools from a large corpus and generates sequences of tool calls to produce final answers, along with a custom API registration system that lets users add their own REST endpoints for the model to discover and invoke. The platform includes a complete instruction-tuning pipeline for training models on curated tool-use data, a multi-tool execution engine that coordinates sequ

KServe is an open platform for deploying and serving generative and predictive AI models on Kubernetes. It defines inference services as custom resources with declarative YAML specifications, enabling a Kubernetes-native approach to model deployment and lifecycle management. The platform leverages Knative-based serverless scaling for automatic scale-to-zero and revision management, and supports a pluggable serving runtime architecture that maps model formats to containerized execution environments. KServe distinguishes itself through model-aware autoscaling that scales replicas based on token

Acest proiect este site-ul oficial de documentație Kubernetes, servind drept resursă tehnică cuprinzătoare pentru gestionarea aplicațiilor containerizate. Funcționează ca un portal de documentație tehnică open-source care oferă ghiduri, tutoriale și materiale de referință pentru software-ul de sisteme distribuite. Site-ul este construit folosind un generator de site-uri statice cu o arhitectură de template-uri bazată pe componente pentru a menține tipare de design consistente. Dispune de un generator de documentație OpenAPI care parsează specificațiile tehnice pentru a construi și actualiza automat pagini de referință API structurate. Pentru a susține o audiență globală, utilizează rutarea conținutului conștientă de internaționalizare pentru a gestiona versiunile localizate ale manualelor. Fluxul de lucru de dezvoltare include un server cu hot-reloading pentru previzualizarea modificărilor site-ului și randarea țintită a limbajului pentru a accelera timpii de build. Proiectul acoperă o gamă largă de domenii tehnice, inclusiv orchestrarea clusterelor, configurarea rețelei și gestionarea resurselor.