Projects sharing features with DeepSeek V2

DeepSeek-LLM is a large language model and causal language model designed for natural language generation. It functions as a multi-lingual system capable of predicting the next token in a sequence to perform text completion and conversational generation. The model is specialized for logical reasoning, specifically as a code and math LLM. This enables it to perform complex problem solving, which includes generating executable code and solving mathematical equations through step-by-step analysis. The system's broader capabilities cover conversational AI, including the generation of chat comple

InternLM is a large language model and a comprehensive suite of weights designed for text generation and complex reasoning. It functions as an inference engine for serving responses, a fine-tuning framework for adjusting model weights, and a platform for building autonomous AI agents. The system is capable of processing long-context input sequences up to one million tokens for document analysis. It employs chain-of-thought reasoning to solve knowledge-intensive tasks by generating intermediate logic steps before producing a final answer. The project covers model weight optimization through s

Llama 3 is a collection of pretrained, autoregressive transformer-based models designed for natural language generation, reasoning, and complex instruction following. It functions as a generative AI framework that provides the infrastructure for managing model weights, executing neural network inference, and handling computational workloads across diverse knowledge domains. The project distinguishes itself through an integrated AI safety toolkit that employs secondary classification filtering to inspect inputs and outputs, ensuring adherence to usage compliance and safety standards. It suppor

Grok-1 is an open-weights large language model implementation featuring a sparse mixture-of-experts architecture. It is designed for high-performance text generation and natural language processing by activating only a subset of specialized expert layers per token. The model utilizes 8-bit weight quantization to reduce memory overhead and accelerate loading. To manage its high parameter count, the implementation supports activation sharding, which distributes the memory load across multiple hardware devices during execution. The project covers large-scale model inference, including text comp

gpt-neox is a distributed training system and framework for building large-scale autoregressive language models. It implements the transformer architecture and provides a toolkit for training models with billions of parameters by distributing weights across compute clusters. The framework distinguishes itself through extensive support for distributed model parallelism, including pipeline and sequence parallelism, to overcome single-device memory limits. It further supports sparse model architectures using a mixture of experts system with Sinkhorn-based routing. The project covers a broad ran

Qwen3-Coder is a specialized large language model designed for software development, technical reasoning, and automated code synthesis. Built on transformer-based sequence modeling, it functions as a multilingual programming assistant capable of generating, completing, and debugging source code across more than one hundred programming languages. The model distinguishes itself through its capacity to process and maintain logical coherence across massive datasets, supporting context windows of up to one million tokens. This allows for repository-scale reasoning, enabling the model to analyze co

OpenMythos is a framework for implementing recurrent large language model architectures. It utilizes recurrent transformer blocks to enable compute-adaptive reasoning and variable processing depth through multiple iterative passes over the same weights. The system features a mixture of experts framework that routes tokens between shared and specialized layers to optimize parameter usage. It also includes parameter-efficient fine-tuning tools using low-rank adaptation modules to modify model behavior with minimal weight updates. The framework covers distributed training pipelines using data p

This project is a comprehensive technical course study guide and reference for learning the architectures and training methods of Transformers and large language models. It serves as a technical overview for understanding how neural networks process data and how to align model behavior with specific performance goals. The repository provides specialized guides on several key areas of model development. This includes detailed references for transformer architectures, implementation frameworks for retrieval-augmented generation and agentic workflows, and technical guides for model optimization

tiny-llm is a large language model inference engine and transformer model implementation. It serves as a quantized model runtime and paged key-value cache manager, providing a specialized inference stack optimized for Apple Silicon. The system distinguishes itself through high-throughput execution techniques, including continuous batching and paged attention. It utilizes a paged memory system to eliminate fragmentation during token generation and employs on-the-fly dequantization of compressed weights to reduce the memory footprint during matrix multiplication. The project covers a broad ran

Sglang is a high-performance inference engine and serving system designed for large language and multimodal models. It provides a programmable interface for orchestrating complex generation workflows, enabling developers to coordinate multi-turn dialogues, tool invocations, and reasoning chains through a domain-specific language. The platform is built to support production-scale deployments, offering an OpenAI-compatible API that allows for integration with existing application ecosystems. The system distinguishes itself through a disaggregated architecture that separates compute-intensive pr

Baichuan-7B is an open-source 7 billion parameter bilingual Transformer model designed for text generation and few-shot learning across Chinese and English. It is built on a large Transformer architecture trained on a bilingual corpus, enabling it to produce coherent text in both languages from a single model. The model incorporates several optimization techniques that distinguish it from standard large language models. It uses rotary position embeddings that can extrapolate to longer sequences than seen during training, allowing context extension beyond the original 4096-token training lengt

Long Llama is a transformer-based language model and fine-tuning framework designed to process and maintain logical coherence across input sequences that significantly exceed standard length limits. By utilizing a focused transformer architecture, the project enables models to handle massive documents or entire books by training attention layers to track distant tokens. The framework distinguishes itself through specialized attention mechanisms that allow for the processing of hundreds of thousands of tokens. It incorporates memory-efficient inference techniques, such as key-value caching and

ChatGLM2-6B is an open-weight large language model designed for natural language conversations and text generation in both English and Chinese. It functions as a bilingual chat model capable of processing and maintaining coherence across text sequences up to 32K tokens. The model is optimized for local deployment through precision quantization, which reduces memory requirements to allow execution on consumer-grade hardware. It supports distributing model weights across multiple graphics cards to handle parameters that exceed the memory of a single device. The project covers capabilities for

This project provides a Chinese large language model based on the LLaMA architecture. It is an instruction-tuned model optimized for natural language processing and multi-turn conversations in Chinese. The system includes a framework for parameter-efficient fine-tuning using low-rank adaptation and quantization to reduce memory requirements. It also implements retrieval augmented generation for local document question answering and supports long-context processing for sequences up to 64K tokens. The project covers a broad set of capabilities including supervised instruction tuning, reinforce

Yi is a bilingual language model and foundation model designed for natural language processing, reasoning, and reading comprehension in both English and Chinese. It is built as a transformer-based architecture capable of general purpose text generation and conversational tasks. The model is distinguished by its ability to function as a long context system, processing and analyzing extended input sequences up to 200k tokens. It also supports quantized versions that use low-bit precision to reduce memory footprints, enabling execution on consumer-grade hardware. The project covers a broad rang

Transformers is a comprehensive library for machine learning that provides a unified interface for training, fine-tuning, and deploying transformer-based models. It supports a wide range of tasks, including text classification, language modeling, question answering, and sequence-to-sequence translation, while offering specialized architectures for both text and vision processing. The framework includes tools for managing the entire model lifecycle, from data preprocessing and tokenization to distributed training and inference. The library features extensive support for model optimization and

The official repo of MiniMax-Text-01 and MiniMax-VL-01, large-language-model & vision-language-model based on Linear Attention

ChatGLM-6B is an open-source bilingual large language model designed for natural dialogue and text generation in both English and Chinese. It is structured as a dialogue model capable of tasks such as role-playing and information extraction. The project provides implementations for quantized language models, using low-precision weights to reduce GPU memory requirements for local inference. It also supports parameter-efficient fine-tuning, allowing model behavior to be optimized for specific tasks without requiring full retraining. The model includes capabilities for local execution on GPUs a

Fast-GraphRAG is a system for generating and querying knowledge graphs from domain data. It uses a GraphRAG retrieval workflow to traverse structured data and isolate precise evidence for answering complex questions. The project utilizes an agent-driven retrieval framework to coordinate the querying of knowledge graphs and the synthesis of final answers. It supports incremental data synchronization, allowing structured knowledge bases to be updated in real time as source information evolves. The system integrates with API-compatible language models and embedding providers to power its data p

MergeKit is a toolkit for combining multiple pre-trained large language models into a single entity using algorithmic blending. It provides a specialized system for parameter interpolation and weight extraction to unify model capabilities. The project distinguishes itself through an evolutionary merge optimizer that tunes parameters based on quantitative evaluation metrics. It also features a mixture of experts orchestrator capable of converting dense models into sparse architectures and a tokenizer alignment tool for transplanting embeddings between different models. The toolkit covers a br

FlashInfer is a library of high-performance GPU kernels purpose-built for accelerating large language model inference. It provides optimized implementations for attention operations (including flash attention, page attention, multi-head latent attention, and cascade attention) using paged key-value caches, fused kernel composition, and just-in-time compilation. The library also includes specialized kernels for mixture-of-experts layers, block-scaled low-precision quantization (FP8, FP4), and distributed collective communication. What distinguishes FlashInfer is its fused all-reduce communicat

Qwen is a comprehensive framework for large language model development, serving, and deployment. It provides a complete ecosystem for transformer-based sequence modeling, offering base models alongside specialized tools for instruction-tuned alignment, fine-tuning, and long-context inference. The project is designed to support both research and production environments, enabling users to train, optimize, and host generative models locally or across distributed hardware. The framework distinguishes itself through its focus on high-performance serving and extensibility. It features a high-perfor

This project is a manual reconstruction of the Llama 3 transformer architecture implemented as a PyTorch neural network. It serves as a reference for the internal mathematical structure and tensor flow of a transformer-based language model designed for next token prediction. The implementation focuses on building the model from scratch using basic matrix operations and tensor manipulations. It demonstrates the manual construction of core components, including rotary positional embeddings, multi-head self-attention, and root mean square normalization. The codebase covers the full inference pi

Mergekit is a toolkit for combining multiple pretrained large language models into a single model. It functions as an architecture assembler and merging system that transfers capabilities between models using weighted algorithms and layer-wise assembly without requiring additional training. The project provides specialized utilities for extracting low-rank approximations from fine-tuned models to create portable parameter updates. It also includes a framework for converting dense language models into a mixture of experts architecture by constructing gating mechanisms to route inputs to specia

nlp-recipes is a collection of implementation guides and reference templates for applying natural language processing techniques to real-world tasks. It provides standardized workflows and code examples for developing NLP pipelines, from dataset preparation and model training to performance evaluation. The project focuses on the practical application of transformer-based models, offering patterns for fine-tuning pretrained architectures for tasks such as text classification, named entity recognition, and question answering. It also includes a toolkit for model interpretability, allowing users

Mistral Inference is a library for running Mistral large language models on a GPU, generating text from prompts with token streaming. It loads pretrained model weights from local disk or a remote registry into GPU memory, then produces output tokens one by one for real-time display in interactive applications. The library supports multimodal prompts that accept image URLs alongside text, enabling visual description and reasoning. It includes content safety guardrails that scan generated text against predefined policies to block or flag policy violations. For structured interactions, it provid

fastllm is a set of specialized software components for model weight conversion, Mixture-of-Experts runtimes, and tensor parallelism. It provides an OpenAI compatible API server to expose large language model capabilities through a standardized request format. The project features a tensor parallelism framework that splits computational workloads across multiple GPUs to accelerate execution. It includes a dedicated runtime optimized for Mixture-of-Experts architectures and a quantization tool to convert model weights into lower precision formats to reduce memory usage and increase throughput.