Awesome GitHub RepositoriesModel Sparsification

pykan is a library for implementing Kolmogorov-Arnold Networks, replacing fixed node activation functions with learnable spline functions located on the network edges. It serves as an interpretable AI framework and symbolic regression tool designed to derive transparent mathematical rules from complex data. The project focuses on converting learned numerical functions into human-readable symbolic expressions through library matching and formula conversion. It utilizes additive-compositional topologies and learnable piecewise polynomial segments to approximate non-linear mappings. The framewo

PowerInfer is an inference engine and serving framework designed to run large language models on local hardware. It combines a hybrid CPU-GPU offloader, a quantization tool, and a sparse model optimizer to enable the execution of high-parameter models on consumer-grade devices. The system distinguishes itself through neuron-activation-based offloading, using a predictor model to preload frequent neurons into VRAM while keeping rare neurons in system memory. This hybrid execution model balances workloads between the GPU and CPU based on input patterns to optimize memory access and increase tok

MergeKit is a toolkit for combining multiple pre-trained large language models into a single entity using algorithmic blending. It provides a specialized system for parameter interpolation and weight extraction to unify model capabilities. The project distinguishes itself through an evolutionary merge optimizer that tunes parameters based on quantitative evaluation metrics. It also features a mixture of experts orchestrator capable of converting dense models into sparse architectures and a tokenizer alignment tool for transplanting embeddings between different models. The toolkit covers a br

x-transformers एक PyTorch लाइब्रेरी और रिसर्च टूलकिट है जिसे ट्रांसफॉर्मर आर्किटेक्चर बनाने के लिए बनाया गया है। यह प्रयोगात्मक ट्रांसफॉर्मर रिसर्च को लागू करने के लिए एक मॉड्यूलर फ्रेमवर्क प्रदान करता है, जिसमें उन्नत अटेंशन मैकेनिज्म, लॉन्ग-सीक्वेंस मॉडलिंग टूल्स और विज़न ट्रांसफॉर्मर्स के लिए एक फ्रेमवर्क शामिल है। यह प्रोजेक्ट मेमोरी-एफिशिएंट और हाई-परफॉरमेंस कंपोनेंट्स पर केंद्रित है, जैसे कि टाइल्ड कर्नेल के साथ Flash Attention और मल्टी-क्वेरी अटेंशन। यह कॉन्टेक्स्ट विंडोज़ का विस्तार करने के लिए विशेष तरीके भी लागू करता है, जिसमें सीक्वेंस रिकरेंस और रोटरी पोजीशनल एम्बेडिंग्स शामिल हैं। यह लाइब्रेरी ट्रेनिंग को स्थिर करने के लिए विभिन्न नॉर्मलाइजेशन स्कीम्स, गेटेड फीडफॉरवर्ड नेटवर्क्स और Macaron नेटवर्क्स जैसे कस्टम लेयर टोपोलॉजी सहित आर्किटेक्चरल क्षमताओं की एक विस्तृत श्रृंखला को कवर करती है।

This project is a PyTorch library for building and training Kolmogorov-Arnold Networks. It implements a neural network architecture that replaces fixed activation functions with learnable spline-based functions on edges, serving as a tool for interpretable machine learning. The implementation utilizes reformulated matrix operations to reduce memory overhead and increase computation speed. It employs L1 regularization to sparsify network weights, which improves the transparency of the model's internal logic and decisions. The framework covers a range of capabilities including grid-based funct

Model-Optimizer is a deep learning toolkit and framework dedicated to compressing, pruning, quantizing, and optimizing neural network architectures. It provides methodologies covering weight quantization, model distillation, and speculative decoding for efficient text generation, alongside automated neural architecture search for discovering optimal network structures. The library implements post-training quantization pipelines that convert high-precision neural network weights into lower-bit formats using calibration data. Additional optimization techniques include teacher-student knowledge

PocketFlow is an integrated toolkit for deep learning model compression, distributed training, and mobile format optimization. It provides a system for reducing the size and complexity of neural networks to improve inference efficiency, featuring a dedicated engine for knowledge distillation and a mobile model optimizer. The framework differentiates itself through an automated hyperparameter tuning system that uses reinforcement learning and statistical models to determine optimal compression ratios and layer-wise bit allocation. It also includes a distributed training system that utilizes mu