Awesome GitHub RepositoriesLong-Context Sequence Processors

Qwen3-Coder is a specialized large language model designed for software development, technical reasoning, and automated code synthesis. Built on transformer-based sequence modeling, it functions as a multilingual programming assistant capable of generating, completing, and debugging source code across more than one hundred programming languages. The model distinguishes itself through its capacity to process and maintain logical coherence across massive datasets, supporting context windows of up to one million tokens. This allows for repository-scale reasoning, enabling the model to analyze co

TensorRT-LLM is a platform and toolkit designed for compiling, optimizing, and serving transformer-based models on accelerated hardware. It functions as a framework that transforms machine learning models into efficient execution graphs, providing an engine to refine these models for specific hardware to maximize throughput and minimize latency during text generation. The project distinguishes itself through advanced execution strategies that manage the entire inference pipeline. It utilizes kernel-level fusion and static graph execution to optimize mathematical operations and computational f

Airllm is a framework designed to execute and fine-tune large language models on consumer-grade hardware. By employing layer-wise model decomposition and memory-efficient loading techniques, the engine enables the operation of massive models that would otherwise exceed available system or video memory. The project distinguishes itself through a suite of optimization strategies that balance memory footprint with performance. It utilizes block-wise weight quantization and asynchronous layer prefetching to reduce resource consumption and hide data transfer latency. Additionally, the framework su

Lingbot-map is a feed-forward neural network designed for real-time 3D scene reconstruction from streaming video. It processes video frames one at a time without iterative optimization, producing dense geometry and camera poses at interactive frame rates directly from a live feed. The project distinguishes itself through its ability to maintain stable geometry and pose alignment across very long video sequences, handling thousands of frames without drift. It achieves this through a combination of coordinate grounding memory, sliding-window inference with overlapping keyframes, and a paged KV

bert4keras ist eine leichtgewichtige Reimplementierung der BERT-Transformer-Architektur für das Keras-Deep-Learning-Framework. Es dient als Toolkit für die natürliche Sprachverarbeitung (NLP) und als Transformer-Modellbibliothek, die für Textklassifizierung, Sequenz-Labeling und die Extraktion semantischer Embeddings verwendet wird. Das Framework enthält ein Sequence-to-Sequence-Modellsystem für Question-Answering und Textgenerierung sowie einen Modell-Inference-Server, um trainierte Transformer als Web-APIs für Echtzeit-Vorhersagen bereitzustellen. Die Funktionen decken ein breites Spektrum an Aufgaben des natürlichen Sprachverständnisses ab, einschließlich Leseverständnis, Relationsextraktion und Langtextverarbeitung. Die Bibliothek bietet Tools für das Pre-Training und Fine-Tuning von Sprachmodellen sowie Optimierungstechniken wie Parameterreduktion, adversarielles Training für Robustheit und schichtweise Konfiguration der Lernrate. Das Projekt enthält einen Weight-Conversion-Loader, um vortrainierte Gewichte aus externen Formaten in kompatible Keras-Strukturen umzuwandeln.

Flash Linear Attention is a training framework and inference engine for sequence models that use linear attention and state space mechanisms, designed to process long contexts with reduced memory and compute overhead. It provides hardware-optimized token mixing layers and fused CUDA kernels that minimize memory bandwidth and launch overhead across different GPU architectures, and includes a causal inference engine that generates text token-by-token using cached hidden states for efficient autoregressive decoding. The project supports building hybrid sequence models that interleave standard at

DeepSeek-V2 ist ein Large Language Model, das für die Verarbeitung natürlicher Sprache und die Analyse langer Textsequenzen entwickelt wurde. Es nutzt eine Mixture-of-Experts-Architektur, um hohe Leistung mit effizienter Inferenz zu kombinieren. Das Modell verwendet einen Sparse-Routing-Mechanismus und geteilte Expert-Neuronen, um allgemeines Wissen zu erfassen und gleichzeitig Spezialisierungen beizubehalten. Durch Multi-Head Latent Attention, Group-Query Attention und Low-Rank-Tensor-Kompression werden der Speicherbedarf reduziert und der Durchsatz erhöht. Diese Funktionen ermöglichen die Verarbeitung und den Abruf von Informationen aus umfangreichen Token-Mengen und unterstützen einen wirtschaftlichen Betrieb durch geringere Hardwarekosten und weniger Speicherengpässe. Das System ist mit Standard-API-Schnittstellen kompatibel und lässt sich in bestehende Toolchains für Sprachmodelle integrieren.

mini-sglang is a collection of tools for large language model inference, serving as an OpenAI-compatible inference server, a memory-efficient prefill engine, and a tensor parallelism runtime. It also functions as a local batch processing engine for offline benchmarking and ablation studies. The project focuses on acceleration and memory management through a KV cache manager that reuses precomputed caches for shared request prefixes. It handles large model workloads by distributing tasks across multiple GPUs and manages peak memory consumption by splitting long input sequences into smaller chu