How this analysis was created: This summary and feature list are AI-generated from collected project material and can contain mistakes. Stars, license and language are imported from GitHub. Inclusion does not mean that we have tested or audited this project. Check the source documentation for any feature you depend on. Learn more on our About page.
StableLM is a pre-trained transformer-based large language model designed for natural language generation and zero-shot inference. It functions as a causal language model that predicts the next token in a sequence to produce human-like text for conversational and creative writing tasks. The model is built as a fine-tunable base, allowing the adaptation of pre-trained weights to specific tasks or styles through custom dataset training and weight regularization. It utilizes rotary positional embeddings and flash-attention to optimize memory usage and processing efficiency during deployment on G
FlashAttention is an attention mechanism optimization library and machine learning acceleration framework designed to increase training speed and reduce memory footprint for large-scale neural network models. It functions as a collection of low-level CUDA kernels that optimize memory-bound operations to improve hardware utilization on graphics processing units. The library distinguishes itself through an input-output-aware algorithm design that minimizes data movement between different levels of memory. By employing kernel fusion and tiled matrix multiplication, it combines sequential operati
3FS is a distributed file system and RDMA storage cluster designed for high-performance AI training and inference workloads. It functions as a strongly consistent storage layer that utilizes a disaggregated architecture to pool SSDs and memory resources across multiple nodes. The system provides specialized storage implementations including an AI training checkpoint store for parallel state preservation and a distributed key-value cache store for decoder layer vectors to optimize inference processing. It ensures data integrity through chain replication and apportioned query distribution. The
ICML 2025 Spotlight ShadowKV: KV Cache in Shadows for High-Throughput Long-Context LLM Inference
The repository is the PyTorch implementation of HyperAttention paper:
The main features of insuhan/hyper-attn are: Attention Optimization.
Projects with overlapping indexed features include: stability-ai/stablelm — StableLM is a pre-trained transformer-based large language model designed for natural language generation and… dao-ailab/flash-attention — FlashAttention is an attention mechanism optimization library and machine learning acceleration framework designed to… deepseek-ai/3fs — 3FS is a distributed file system and RDMA storage cluster designed for high-performance AI training and inference… deepseek-ai/deepep — DeepEP is a distributed model accelerator and expert-parallel communication library designed to optimize the training… deepseek-ai/deepgemm — DeepGEMM is a suite of specialized GPU kernels and a just-in-time compiler designed for low-precision matrix… bytedance/shadowkv — [ICML 2025 Spotlight] ShadowKV: KV Cache in Shadows for High-Throughput Long-Context LLM Inference.