7 Repos
Awesome GitHub RepositoriesModel Optimization
Tools for compressing, quantizing, and standardizing models.
Explore 7 awesome GitHub repositories matching part of an awesome list · Model Optimization. Refine with filters or upvote what's useful.
Awesome Model Optimization GitHub Repositories
Finde die besten Repos mit KI.Wir suchen mit KI nach den am besten passenden Repositories.
- onnx/onnx
onnx/onnx
20,358Auf GitHub ansehenONNX is an open-source standard for machine learning interoperability that provides a unified format for representing neural network models. By defining a common set of operators and a standardized file structure, it enables models to be shared, exported, and executed consistently across different training frameworks and software ecosystems. The project functions as an intermediate representation layer that decouples model development from deployment. It utilizes a language-neutral binary serialization format to store model structures and weights, ensuring that computational graphs remain por
Open format for model interoperability across frameworks.
Pythonaiartificial-intelligencedeep-learning - ggml-org/ggml
ggml-org/ggml
13,985Auf GitHub ansehenGGML is a machine learning tensor library and neural network engine written in C. It functions as a compute-focused runtime designed to execute transformer-based models and perform complex mathematical operations on multi-dimensional arrays directly on local consumer hardware. The library distinguishes itself by enabling local inference for large language models and edge machine learning deployment without reliance on external cloud infrastructure. It achieves this through a tensor-based computation graph that organizes operations for efficient execution and memory management, alongside stati
High-performance tensor library for CPU-based inference.
C++automatic-differentiationlarge-language-modelsmachine-learning - autogptq/autogptq
AutoGPTQ/AutoGPTQ
5,070Auf GitHub ansehenAutoGPTQ ist ein Toolkit zur Modellkomprimierung und ein Framework zur Post-Training-Quantisierung, das entwickelt wurde, um den Speicherbedarf großer Sprachmodelle zu reduzieren. Es nutzt den GPTQ-Algorithmus zur Komprimierung neuronaler Netzwerkgewichte, wodurch Hardwareanforderungen gesenkt und die VRAM-Nutzung reduziert werden. Das Projekt dient als Inferenz-Beschleuniger durch die Bereitstellung optimierter Kernels, die die Token-Generierungsgeschwindigkeit erhöhen. Es bietet Erweiterbarkeit der Modellarchitektur, wodurch Quantisierungsfunktionen durch konfigurierbare Muster zu neuen Modellstrukturen hinzugefügt werden können. Das Framework deckt eine umfassende Quantisierungspipeline ab, einschließlich schichtweiser Gewichtskomprimierung, kalibrierungsbasierter Skalenschätzung und präzisionsspezifischem Memory-Mapping. Es enthält zudem Systeme zur Bewertung der Modellperformance, um die Auswirkungen der Quantisierung auf die Genauigkeit bei Sprach- und Zusammenfassungsaufgaben zu messen.
Quantization package for LLMs based on GPTQ.
Python - mit-han-lab/llm-awq
mit-han-lab/llm-awq
3,563Auf GitHub ansehenMLSys 2024 Best Paper Award AWQ: Activation-aware Weight Quantization for LLM Compression and Acceleration
Technique for weight quantization and model acceleration.
Python - intel/neural-compressor
intel/neural-compressor
2,585Auf GitHub ansehenNeural Compressor is a deep learning model compression toolkit and AI inference acceleration engine. It functions as an automated model quantization tool and hardware-aware model compiler designed to reduce the memory footprint of neural networks and decrease execution latency. The project provides specialized frameworks for optimizing large language models, utilizing weight-only quantization and hardware-specific kernels to improve the operational efficiency of generative AI workloads. It maps neural network operators to specialized CPU and GPU vector instructions to accelerate model executi
Toolkit for model compression, pruning, and distillation.
Pythonauto-tuningawqfp4 - casper-hansen/autoawq
casper-hansen/AutoAWQ
2,342Auf GitHub ansehenAutoAWQ implements the AWQ algorithm for 4-bit quantization with a 2x speedup during inference. Documentation:
Package for 4-bit quantization of language models.
Python - huggingface/optimum-quanto
huggingface/optimum-quanto
1,043Auf GitHub ansehenA pytorch quantization backend for optimum
Library for simplifying deep learning model quantization.
Python
