Awesome GitHub RepositoriesModel Orchestrators

Fooocus is a generative image interface designed to simplify the creation of high-quality visual content from text descriptions. It functions as a latent diffusion pipeline and model orchestrator, managing the complex interactions between neural network layers, mathematical samplers, and hardware resource allocation to produce professional-grade imagery. The project distinguishes itself through a sophisticated prompt engineering engine and modular style management. Users can dynamically modify output characteristics by injecting style adapters directly into prompts or by utilizing wildcards a

Cherry Studio is a cross-platform desktop application that serves as a centralized workspace for managing and interacting with multiple artificial intelligence models. It functions as a local-first orchestrator, prioritizing user privacy by storing all conversation history and knowledge bases directly on your device. By providing a unified interface for both cloud-based and local AI services, the platform simplifies API key management and allows for consistent model interaction across different operating systems. The application distinguishes itself through a robust retrieval-augmented genera

DeepSpeed is a distributed deep learning optimization library and framework designed for the training and inference of massive AI models. It serves as a model parallelism orchestrator and a toolkit for scaling large language models across multiple GPUs and compute nodes. The project distinguishes itself through 3D parallelism orchestration, which combines data, pipeline, and tensor parallelism. It utilizes ZeRO-based memory partitioning to eliminate redundant storage and employs CPU-offload memory management to move weights and optimizer states to system RAM. Additionally, it provides special

One API is a centralized gateway and orchestration platform designed to consolidate multiple artificial intelligence model providers into a single, standardized interface. It functions as a reverse proxy that intercepts incoming API requests and routes them to various third-party services, abstracting the underlying provider credentials through a unified token management system. The platform provides comprehensive administrative tools for managing API keys, rotating credentials, and enforcing security policies across diverse service integrations. It includes a persistent database-backed syste

Modular is a unified machine learning development platform designed for building, compiling, and deploying high-performance neural network models. It provides a comprehensive execution engine that supports both local and production-grade inference, enabling developers to manage the entire model lifecycle from initial architecture definition to scalable, containerized service deployment. The platform distinguishes itself through a hardware-agnostic runtime that abstracts diverse silicon architectures, allowing models to execute efficiently across varied compute environments. It includes a spec

Dia is a generative AI audio tool and text-to-speech synthesis engine designed for the production-ready deployment of machine learning models. It provides a framework for creating lifelike synthetic speech by conditioning generation on reference audio samples to replicate specific vocal characteristics, emotional tones, and delivery styles. The system distinguishes itself through its ability to perform custom voice cloning and precise control over audio output. Users can adjust generation parameters such as temperature and guidance scale to modify the pacing, creativity, and style of the synt

Ragas is an evaluation framework designed to measure the performance of retrieval-augmented generation pipelines and autonomous agent workflows. It provides a comprehensive suite of tools for benchmarking system outputs, utilizing language models as automated judges to score performance against defined rubrics and reference data. By standardizing inputs, retrieved contexts, and generated responses into a unified schema, the project enables consistent analysis across complex AI applications. The framework distinguishes itself through its ability to generate synthetic test datasets from existin

FlexGen is an inference engine for large language models designed for high-throughput execution on single or multiple GPUs. It functions as a framework for managing model execution through a combination of memory offloading, weight compression, and pipeline orchestration. The system enables the execution of models that exceed available GPU memory by moving tensors and caches between GPU memory, system RAM, and disk storage. It utilizes 4-bit weight quantization to reduce the memory footprint of model parameters, allowing for increased batch processing capacity. The project covers distributed

h2o-3 is a distributed machine learning platform and automated machine learning framework designed for training and deploying predictive models using distributed in-memory computing. It functions as a deep learning framework and a distributed model scoring engine, capable of operating as a Kubernetes ML cluster to process large datasets in parallel. The platform distinguishes itself through automated machine learning capabilities that automatically select the best algorithms and hyperparameters to optimize model performance. It provides specialized deep learning toolkits for tasks including i

Bella is an AI companion system featuring a conversational interface for interacting with local and cloud artificial intelligence models. It integrates a local model manager to automate the download and organization of machine learning weights and a speech-to-text transcription engine to enable hands-free interaction. The project includes an emotion visualization system that uses cross-fading video playback to represent the agent's internal state. This is driven by a state-driven emotion mapping system and an interaction-based affinity system that tracks user engagement frequency to trigger s

Torchtitan is a reference implementation for distributed deep learning built within the PyTorch ecosystem. It provides a framework for training large neural network models across multiple GPUs and nodes by combining several parallelism techniques, including fully sharded data parallelism (FSDP), tensor parallelism, and pipeline parallelism, making it possible to train models that exceed the memory capacity of a single device. The system distinguishes itself through asynchronous checkpointing, which saves model and optimizer state to persistent storage without pausing the training loop, enabli

Ce projet est un guide architectural MLOps et un framework pour concevoir et déployer des systèmes d'apprentissage profond dans des environnements de production. Il fournit une approche structurée pour le déploiement de l'inférence de modèles, l'orchestration de pipelines ML et la création d'architectures de machine learning de niveau production. Le projet se distingue par un accent mis sur l'apprentissage profond distribué et l'IA en périphérie (edge AI). Il couvre des méthodologies pour paralléliser l'entraînement des modèles sur plusieurs GPU afin de gérer de grands jeux de données et applique des techniques comme la quantification et la distillation pour réduire la taille des modèles pour le matériel embarqué. La surface de capacité s'étend à la surveillance et à l'observabilité, incluant le suivi de la performance des modèles, la dérive des données et les métriques d'expérience. Il aborde également l'orchestration des flux de travail de données, le versionnage des jeux de données via des magasins d'objets, et la gestion des requêtes d'inférence à haut volume en utilisant le traitement par lots adaptatif et l'orchestration basée sur des conteneurs.

Superlinked is a development framework designed for building semantic search and retrieval pipelines. It functions as a machine learning data pipeline and semantic retrieval engine, providing the tools necessary to unify data schema definition, embedding generation, and vector database integration within a single application. The framework distinguishes itself by acting as a vector database orchestrator that manages the lifecycle of machine learning models alongside complex search logic. It enables developers to construct structured data models that map raw content and metadata into unified r