ML frameworks and MLOps

DeepSeek-V3 is a large language model that provides comprehensive resources for model utilization, including technical specifications, pre-trained weights, and evaluation benchmarks. The project details the core transformer architecture, including parameter counts and multi-token prediction modules, while supporting native 8-bit floating-point quantization. The repository offers extensive support for local and distributed inference through integration with multiple frameworks and engines. It includes documentation for deploying the model across various hardware configurations, such as GPUs and specialized accelerators, alongside guidance for environment setup and weight conversion. Users can access the model through a web-based chat interface, a programmatic API, or by downloading weights for local execution.

Cog is a machine learning packaging tool and containerized model wrapper that bundles models and their dependencies into standardized Docker containers. It functions as an environment manager and inference server, ensuring consistent model execution across different hardware systems by resolving GPU drivers, system libraries, and Python dependencies. The project distinguishes itself by automatically generating RESTful HTTP servers and OpenAPI schemas based on defined model input and output types. It manages large model weights as external fixtures to optimize image size and utilizes a slot-based system to control concurrency and optimize hardware resource utilization. The platform covers a broad range of capabilities, including model training workflows, local model execution for testing, and image registry integration for deployment. It also provides monitoring utilities for health checks and log routing, as well as security features for secret input integration and sensitive data redaction.

Ultralytics is a comprehensive computer vision framework designed for training, validating, and deploying deep learning models across a wide range of visual recognition tasks. It provides a unified interface for core operations including object detection, instance segmentation, pose estimation, and image classification. By utilizing a modular architecture, the platform allows users to swap model components to balance inference speed and accuracy requirements for diverse applications. The framework distinguishes itself through its support for real-time processing and flexible deployment. It includes a streaming inference engine that manages memory usage for large-scale video analysis and a format-agnostic export pipeline that translates trained weights into standardized formats for edge and cloud environments. Beyond standard detection, it supports open-vocabulary segmentation, allowing users to identify objects using text or visual prompts, and provides robust multi-object tracking capabilities to maintain identity persistence across video frames. The platform covers the entire machine learning lifecycle, from dataset retrieval and dynamic data loading to performance benchmarking and experiment tracking. It includes specialized tools for annotating visual results and accessing structured output data, facilitating integration into automated inspection and monitoring workflows. Users can configure training hyperparameters, resume interrupted sessions, and profile model performance to ensure optimal deployment on hardware ranging from mobile devices to high-performance GPUs.

Burn is a deep learning framework designed for building, training, and deploying neural networks using a modular architecture. As a machine learning library built in Rust, it provides a backend-agnostic computational engine that enables the execution of models across diverse hardware, including central processors, graphics processors, and web runtimes. The framework distinguishes itself through a highly portable design that allows developers to maintain a single workflow for both training and inference across heterogeneous environments. It incorporates advanced optimization techniques such as just-in-time kernel fusion, asynchronous execution, and static graph compilation to maximize computational efficiency and hardware throughput. The library also functions as a comprehensive model quantization toolkit, offering tools to convert weights and activations into lower-bit representations. These capabilities facilitate the deployment of neural networks on resource-constrained edge devices by reducing memory footprints and accelerating inference tasks without requiring manual code changes for different hardware targets.

This repository provides a collection of practical demonstrations and implementation guides for machine learning tasks using TensorFlow.js. It serves as a resource for developers to explore model architectures, training workflows, and data manipulation techniques across domains such as computer vision, natural language processing, and reinforcement learning. The project covers the full lifecycle of machine learning development, including tensor-based mathematical operations, model construction via high-level layer APIs or low-level tensor logic, and model serialization for various storage mediums. It includes utilities for converting models into browser-compatible formats and provides infrastructure for executing these models across diverse backends, including WebGL, WebAssembly, and CPU-accelerated environments. Documentation and examples are organized by task type, allowing users to browse implementations for regression, object detection, and generative models. The repository also includes deployment guides for hosting server-side applications on cloud platforms, alongside tools for managing tensor memory and asynchronous training processes.

Hopsworks - Data-Intensive AI platform with a Feature Store

nanoGPT is a lightweight engine for training and fine-tuning transformer-based language models from scratch. It provides a minimalist codebase designed for educational exploration and rapid experimentation with neural network architectures, utilizing self-attention and feed-forward layers to process sequences and predict subsequent elements. The project distinguishes itself through a focus on high-speed data ingestion and hardware-accelerated performance. It includes a dedicated pipeline for transforming raw text into memory-mapped binary files, which enables efficient streaming during training. To maximize throughput, the system supports distributed data parallelism across multiple graphics processing units and employs just-in-time kernel compilation to optimize mathematical operations for specific hardware. Beyond core training capabilities, the repository provides a command-line interface for generative text inference, allowing users to sample sequences from trained models using configurable parameters. It also includes integrated benchmarking tools to measure iteration speeds and identify hardware bottlenecks, ensuring efficient model development across various configurations.

Open source AI infrastructure layer. Eight agents run on every request: security, caching, memory, hallucination detection, tamper-proof audit trail. One line of code. Your data never leaves your server.

Llama is a computational framework and runtime environment designed for executing transformer-based neural networks locally. It functions as a generative AI inference engine, enabling the processing of input sequences through pre-trained model weights to produce text completions and structured data outputs directly on your own hardware. The system distinguishes itself through specialized memory and computation management techniques, including memory-mapped weight loading and quantization-aware inference, which allow for efficient execution on standard consumer hardware. It utilizes a stateless request execution model and a tensor-based computation graph to handle token-based sequence processing, ensuring that each inference task operates independently without reliance on persistent server state. This project provides the necessary tools for local large language model deployment, including a command-line interface for retrieving authorized model checkpoints and configuration files. It supports offline research and the integration of text generation capabilities into custom software applications, allowing users to manage model parameters such as sequence length and batch size to meet specific performance requirements.

Feast is a machine learning feature store and MLOps data infrastructure layer. It provides a centralized system for managing and serving features across offline training and online production environments, utilizing an online feature serving layer for low-latency retrieval. The project centers on a feature registry that acts as a central catalog for defining, governing, and discovering feature services. It employs a unified data access layer to decouple feature retrieval from physical storage and includes a point-in-time data generator to create historically accurate training datasets that prevent data leakage. The platform covers a broad range of capabilities including real-time model inference, streaming data feature engineering, and the generation of training datasets. It also supports vector embedding search for similarity-based retrieval and feature quality validation to maintain data integrity.

Qwen3 is a transformer-based large language model designed as a generative AI foundation for understanding, reasoning, and generating human language. It functions as a comprehensive ecosystem for model training, fine-tuning, and production-ready inference, providing the underlying architecture and weights necessary to build diverse artificial intelligence applications. The project distinguishes itself through extensive support for model quantization and distributed inference, enabling efficient execution across a wide range of hardware from consumer-grade devices to scalable cloud infrastructure. It includes a specialized toolkit for weight compression and memory optimization, such as key-value cache management, which reduces computational requirements while maintaining performance. Furthermore, the model integrates with agentic frameworks, allowing for the development of autonomous systems capable of executing complex workflows and interacting with external tools. The ecosystem covers a broad surface of deployment and training methodologies, including standardized interfaces for modular plugin integration and function calling. It provides extensive documentation for various training, fine-tuning, and serving environments to facilitate integration into existing software stacks.

AI Infra / AI Orchestration / AI Control Plane

YOLOv5 is a comprehensive computer vision framework designed for end-to-end deep learning, specializing in real-time object detection, image classification, and instance segmentation. It provides a unified toolkit that manages the entire lifecycle of a model, from initial dataset configuration and hyperparameter tuning to high-speed inference and deployment. The framework utilizes a modular neural architecture, allowing users to swap backbone and head components to tailor models for specific visual tasks. What distinguishes this project is its focus on production-ready deployment and model efficiency. It includes a robust model export engine that converts trained networks into standardized formats, enabling high-performance execution across diverse hardware, including edge devices and web browsers. To optimize models for resource-constrained environments, the framework offers advanced techniques such as neural network pruning, weight sparsity, and mixed-precision training, alongside tools for benchmarking performance and fine-tuning pruned models. The platform supports a highly configurable training pipeline that leverages parallel processing and dynamic data augmentation to improve model robustness. Users can manage complex training workflows through externalized configuration files, which decouple model logic from dataset structures. The system also provides sophisticated inference capabilities, including test-time augmentation and model ensembling, to balance detection accuracy with processing latency requirements.

Feast is an open-source feature store for machine learning that provides a central platform for defining, storing, and serving features across both training and inference workflows. It operates as a declarative system where feature definitions are written as code in Python files, synchronized to a central registry, and made available for low-latency online retrieval or point-in-time correct historical joins for training datasets. The project abstracts storage behind a pluggable architecture, allowing offline and online backends to be swapped without changing retrieval logic, and coordinates materialization pipelines that move batch features from offline stores to online stores using configurable compute engines. Feast distinguishes itself through its multi-protocol serving surface, exposing the same feature values simultaneously via REST, gRPC, and MCP protocols to support diverse client ecosystems including AI agents. It includes an on-demand transformation framework that applies Python-based feature transformations at retrieval time, combining precomputed features with request-time data for flexible serving. The project also provides entity-key collocated storage, storing all features for a single entity in one document to reduce online reads to a single lookup per request, and a background registry cache refresh that prevents serving requests from blocking on cache updates. The platform covers the full lifecycle of feature management, including feature engineering and transformation from batch and streaming sources, governance and access control with application-level RBAC and OIDC authentication, real-time inference serving, and historical feature retrieval for training. It supports vector search and retrieval-augmented generation workflows by storing and querying embeddings for similarity search. Feast integrates with a wide range of storage backends, compute engines, and data sources, and provides tooling for deployment on Kubernetes, monitoring with Prometheus and OpenTelemetry, and lineage tracking with OpenLineage.

This project is a comprehensive library of state-of-the-art neural network architectures designed for image classification and feature extraction. It provides a complete deep learning training framework that supports distributed execution, allowing users to build, train, and fine-tune vision models using optimized schedulers and pre-configured training recipes. The library distinguishes itself through a modular backbone architecture that treats neural networks as decoupled feature extractors, enabling the retrieval of multi-scale outputs for downstream tasks like object detection and segmentation. A centralized registry-based model factory allows for the dynamic instantiation of architectures via string identifiers, while externalized hyperparameter files ensure that training workflows remain reproducible. Users can also exercise granular control over the training process through layer-wise optimization configurations and a flexible hook system for intercepting intermediate tensor states. The platform includes extensive utilities for managing the entire lifecycle of a vision model, from data loading and augmentation to inference and deployment. It features a dynamic transformation pipeline that automatically resolves preprocessing requirements based on the chosen model architecture, ensuring that input data is correctly aligned for both training and evaluation. Integration with remote model hubs further facilitates the sharing and retrieval of pre-trained weights and configurations.

🛠 All-in-one web-based IDE specialized for machine learning and data science.

Unsloth is a high-performance training and inference platform designed to optimize the lifecycle of large language and multimodal models. It provides a comprehensive engine for fine-tuning, executing, and managing models locally, with a focus on reducing memory consumption and increasing compute speed on consumer-grade hardware. The platform distinguishes itself through hand-optimized kernels and automated computational graph techniques that maximize hardware throughput. It supports advanced training methodologies, including reinforcement learning for reasoning and efficient adapter-based fine-tuning, while offering a unified web-based interface for no-code model training, data preparation, and real-time performance monitoring. Beyond its core training capabilities, the project includes a local inference runtime that supports API-based deployment, tool-calling, and automated output verification. It manages the entire model development process, from dataset generation and hyperparameter configuration to model exporting and performance benchmarking across diverse hardware configurations. The software provides setup utilities for local development environments and includes diagnostic tools to assist with installation and hardware compatibility.

DVC is a data versioning tool and pipeline orchestrator designed to track large datasets and machine learning models. It functions as a system for managing large data artifacts by storing lightweight metadata in version control while keeping the actual binaries in a separate cache. The project serves as an experiment tracker and remote storage synchronizer, enabling the execution and comparison of machine learning iterations based on hyperparameters and performance metrics. It provides a bridge for pushing and pulling these large data artifacts between local environments and cloud or on-premises storage. The tool covers data pipeline automation through the definition and execution of computational graphs, ensuring only components impacted by changes are rerun. It further supports model reproducibility by reconstructing specific experiment states and syncing the corresponding data and code versions.

vLLM is a high-throughput inference engine designed for the efficient serving and execution of large language models. It functions as a production-ready distributed model server, providing standard API protocols for online serving while also supporting offline batch processing. The system is built to maximize token generation speed and memory efficiency, enabling both large-scale cloud deployments and local execution on personal hardware. The project distinguishes itself through advanced memory management and request scheduling techniques, most notably its use of non-contiguous key-value cache blocks to eliminate fragmentation and its ability to dynamically insert new sequences into batches as they arrive. It provides a hardware-agnostic abstraction layer that maps complex mathematical operations to diverse accelerators, including specialized GPUs and consumer-grade silicon like Apple hardware. This is further supported by custom kernel fusion and a flexible quantization framework that allows for the compression of neural networks to fit resource-constrained environments. Beyond its core runtime, the framework offers extensive support for custom

Lightweight, Python library for fast and reproducible experimentation :microscope: