Awesome GitHub RepositoriesModel Sparsification

pykan is a library for implementing Kolmogorov-Arnold Networks, replacing fixed node activation functions with learnable spline functions located on the network edges. It serves as an interpretable AI framework and symbolic regression tool designed to derive transparent mathematical rules from complex data. The project focuses on converting learned numerical functions into human-readable symbolic expressions through library matching and formula conversion. It utilizes additive-compositional topologies and learnable piecewise polynomial segments to approximate non-linear mappings. The framewo

PowerInfer is an inference engine and serving framework designed to run large language models on local hardware. It combines a hybrid CPU-GPU offloader, a quantization tool, and a sparse model optimizer to enable the execution of high-parameter models on consumer-grade devices. The system distinguishes itself through neuron-activation-based offloading, using a predictor model to preload frequent neurons into VRAM while keeping rare neurons in system memory. This hybrid execution model balances workloads between the GPU and CPU based on input patterns to optimize memory access and increase tok

MergeKit is a toolkit for combining multiple pre-trained large language models into a single entity using algorithmic blending. It provides a specialized system for parameter interpolation and weight extraction to unify model capabilities. The project distinguishes itself through an evolutionary merge optimizer that tunes parameters based on quantitative evaluation metrics. It also features a mixture of experts orchestrator capable of converting dense models into sparse architectures and a tokenizer alignment tool for transplanting embeddings between different models. The toolkit covers a br

x-transformers 是一个用于构建 Transformer 架构的 PyTorch 库和研究工具包。它提供了一个模块化框架来实现实验性 Transformer 研究，包括一套先进的注意力机制、长序列建模工具以及视觉 Transformer 框架。 该项目以专注于内存高效和高性能组件而著称，例如带有分块内核的 Flash Attention 和多查询注意力。它还实现了用于扩展上下文窗口的专门方法，包括序列循环和旋转位置嵌入。 该库涵盖了广泛的架构功能，包括用于稳定训练的各种归一化方案、门控前馈网络以及 Macaron 网络等自定义层拓扑。它支持编码器和解码器构造，为自回归序列生成和图像描述等视觉-语言任务提供工具。

这是一个用于构建和训练 Kolmogorov-Arnold Networks (KAN) 的 PyTorch 库。它实现了一种神经网络架构，用可学习的样条函数替代了边上的固定激活函数，是可解释机器学习的有力工具。 该实现利用重构的矩阵运算来降低内存开销并提高计算速度。它采用 L1 正则化来稀疏化网络权重，从而提高了模型内部逻辑和决策的透明度。 该框架涵盖了从基于网格的函数逼近、B-样条激活函数到深度学习模型优化的多种功能。这些特性均基于原生 PyTorch 张量构建，以支持自动微分和硬件加速。

Model-Optimizer is a deep learning toolkit and framework dedicated to compressing, pruning, quantizing, and optimizing neural network architectures. It provides methodologies covering weight quantization, model distillation, and speculative decoding for efficient text generation, alongside automated neural architecture search for discovering optimal network structures. The library implements post-training quantization pipelines that convert high-precision neural network weights into lower-bit formats using calibration data. Additional optimization techniques include teacher-student knowledge

PocketFlow is an integrated toolkit for deep learning model compression, distributed training, and mobile format optimization. It provides a system for reducing the size and complexity of neural networks to improve inference efficiency, featuring a dedicated engine for knowledge distillation and a mobile model optimizer. The framework differentiates itself through an automated hyperparameter tuning system that uses reinforcement learning and statistical models to determine optimal compression ratios and layer-wise bit allocation. It also includes a distributed training system that utilizes mu