Awesome GitHub RepositoriesSparse Model Architectures

This project is a comprehensive framework for the entire lifecycle of transformer-based language models, supporting everything from foundational pretraining to specialized deployment. It provides a modular toolkit for defining neural network architectures, managing data preparation pipelines, and executing training routines across various scales. The framework is designed to handle the full model development process, including supervised fine-tuning, behavioral alignment, and the integration of agentic capabilities. What distinguishes this framework is its focus on efficient training and adva

Grok-1 is an open-weights large language model implementation featuring a sparse mixture-of-experts architecture. It is designed for high-performance text generation and natural language processing by activating only a subset of specialized expert layers per token. The model utilizes 8-bit weight quantization to reduce memory overhead and accelerate loading. To manage its high parameter count, the implementation supports activation sharding, which distributes the memory load across multiple hardware devices during execution. The project covers large-scale model inference, including text comp

DeepSpeed is a distributed deep learning optimization library and framework designed for the training and inference of massive AI models. It serves as a model parallelism orchestrator and a toolkit for scaling large language models across multiple GPUs and compute nodes. The project distinguishes itself through 3D parallelism orchestration, which combines data, pipeline, and tensor parallelism. It utilizes ZeRO-based memory partitioning to eliminate redundant storage and employs CPU-offload memory management to move weights and optimizer states to system RAM. Additionally, it provides special

PowerInfer is a high-performance local large language model inference engine and sparse inference framework. It provides a runtime for executing models on consumer-grade hardware, utilizing a GPU acceleration backend to optimize tensor operations for graphics processors. The system distinguishes itself through a sparse inference framework that increases generation speed by skipping computations based on activation sparsity in model weights. It includes a GGUF model converter for transforming weights and metadata into a unified binary format, as well as an OpenAI API compatible server for inte

gpt-neox is a distributed training system and framework for building large-scale autoregressive language models. It implements the transformer architecture and provides a toolkit for training models with billions of parameters by distributing weights across compute clusters. The framework distinguishes itself through extensive support for distributed model parallelism, including pipeline and sequence parallelism, to overcome single-device memory limits. It further supports sparse model architectures using a mixture of experts system with Sinkhorn-based routing. The project covers a broad ran

Aerosolve ist ein Machine-Learning-Framework für das Training und Deployment interpretierbarer Modelle. Es dient als Tool für Feature Engineering und Modelltraining, das auf Sparse-Feature-Modellierung setzt, um das Debugging von Gewichten zu vereinfachen und die Dateniteration zu beschleunigen. Das System enthält eine domänenspezifische Transformationssprache, um Rohdaten in modellfertige Repräsentationen zu überführen. Zudem bietet es Funktionen zur visuellen Inhaltsanalyse, indem Bilder in dichte, hochdimensionale Vektorräume abgebildet werden, um Daten nach Stil oder Inhalt zu ordnen und zu klassifizieren. Das Framework ermöglicht ein menschenzentriertes Training, bei dem Vorwissen und spezifische Gewichte direkt in den Lernprozess einfließen können. Für das Deployment nutzt es eine minimale Inference-Runtime für leichtgewichtige Vorhersagen sowie einen Shared-Context-Scoring-Mechanismus zur Verarbeitung mehrerer Elemente in einem einzigen Vorgang.

Engram ist ein dynamisches Wissensabrufsystem und ein Framework zur Speichererweiterung für Large Language Models. Es fungiert als skalierbare Speicher-Lookup-Schicht und spärliche Architekturkomponente, die statisches Modellwissen mit dynamischen externen Zuständen verschmilzt, um die Faktentreue zu verbessern und Halluzinationen zu reduzieren. Das System nutzt konditionalen Speicherabruf und differenzierbare Speicheradressierung, um Eingabe-Token auf spezifische Indizes innerhalb eines assoziativen Speichers im großen Maßstab abzubilden. Dies ermöglicht es dem Modell, seine gesamten verfügbaren Parameter zu erhöhen, indem Gewichte in externen Lookup-Tabellen gespeichert und nur die relevanten Wissenssegmente für eine bestimmte Eingabe aktiviert werden. Das Framework deckt die Optimierung der Modell-Sparsity und skalierbare Erweiterung ab, wobei Key-Value-Retrieval und dynamische Parameterfusion genutzt werden, um die Leistung bei spezialisierten Aufgaben zu steigern, ohne das Netzwerk vollständig neu trainieren zu müssen.

Amazon DSSTNE is a machine learning toolkit and sparse tensor network library designed for deep learning models with sparse inputs and outputs. It provides a model-parallel training framework and a GPU-accelerated sparse engine to support memory-intensive networks. The framework is specifically designed for recommendation system training and large-scale sparse learning. It enables the distribution of large weight matrices and embedding tables across multiple GPU devices to handle models that exceed the memory capacity of a single processor. The project covers a broad range of capabilities in

FastDeploy is a high-performance deployment framework for large language models, vision models, and multimodal models. It provides the infrastructure to launch model services that process combined image, video, and text inputs, exposing these capabilities through a standardized, OpenAI-compatible API for chat and text completions. The project distinguishes itself through advanced inference pipeline engineering and GPU optimization. It employs speculative decoding, tensor parallelism, and a disaggregated execution model that separates prefill and decode phases across different hardware resourc