1 रिपॉजिटरी
Awesome GitHub RepositoriesMixed-Precision Quantizers
Tools that convert model weights into mixed-precision formats to reduce memory footprint.
Distinct from Mixed-Precision Quantization: Focuses on the tool/utility for converting models, whereas the parent is the general technique.
Explore 1 awesome GitHub repository matching artificial intelligence & ml · Mixed-Precision Quantizers. Refine with filters or upvote what's useful.
Awesome Mixed-Precision Quantizers GitHub Repositories
AI के साथ बेहतरीन रिपॉजिटरी खोजें।हम AI का उपयोग करके सबसे सटीक रिपॉजिटरी खोजेंगे।
- turboderp-org/exllamav2
turboderp-org/exllamav2
4,552GitHub पर देखेंexllamav2 is a high-performance inference engine and framework for executing large language models locally on consumer-class GPUs. It provides a complete system for local model deployment, including a specialized inference engine and tools for model quantization. The project features a multi-GPU inference framework that distributes workloads across multiple graphics cards to run models that exceed the memory capacity of a single device. It includes a GPU model quantizer capable of converting models into mixed-precision formats between 2 and 8 bits to balance memory usage and accuracy. The en
Provides a GPU model quantizer for converting models into flexible mixed-precision formats between 2 and 8 bits.
Python
