city96/ComfyUI-GGUF
ComfyUI GGUF
ComfyUI-GGUF is a memory optimizer and model loader for ComfyUI that enables the execution of large transformer-based generative models using quantized weights. It provides a system for loading GGUF formatted weights within a node-based diffusion interface to reduce GPU memory consumption.
The project includes a quantization tool for converting standard model checkpoints into compressed binary formats and a tensor fixer to restore missing keys and correct architectures in binary model files. These utilities ensure that compressed models remain functional during inference on hardware with limited VRAM.
The framework covers model weight optimization and low-memory inference by supporting the loading of quantized diffusion models and text encoders. It manages the process of on-the-fly precision recovery and weight mapping to maintain performance while reducing the total memory footprint.
Features
- ComfyUI Custom Node Suites - Integrates compressed GGUF diffusion models and text encoders as custom nodes within ComfyUI workflows.
- Diffusion Model Memory Optimizers - Provides a framework for running large transformer-based generative models using quantized weights.
- Diffusion Models - Initializes and runs generative image synthesis models based on quantized diffusion architectures.
- GGUF Execution - Implements optimized runtime execution and loading for models using the GGUF quantization format.
- Model Quantization Tools - Handles on-the-fly precision recovery of quantized weights to reduce total memory usage during inference.
- Model Loading - Provides mechanisms for efficiently loading model weights from portable binary formats into memory.
- GGUF Weight Quantization - Provides a utility to convert standard model checkpoints into quantized GGUF artifacts for lower memory usage.
- Quantized Model Implementations - Implements low-precision weight formats to reduce inference memory requirements on limited hardware.
- Weight Dequantization - Enables high-performance inference by recovering weight precision on-the-fly from compressed memory footprints.
- Low-VRAM Inference - Runs large generative models on hardware with limited VRAM by loading quantized GGUF weights.
- Tensor Shape Inferences - Corrects tensor shapes and restores missing keys in model architectures to ensure functional inference.
- Transformer Weight Loading - Optimizes the loading of compressed weights specifically for transformer architectures to reduce initialization overhead.
- Ternary Weight Optimizations - Reduces the memory footprint of transformer architectures to enable faster operation on consumer GPUs.
- Model Architecture Key Restoration - Restores missing architectural keys in model binaries to ensure they remain functional during inference.
- GGUF Format Conversions - Transforms standard model checkpoints into the GGUF format for quantized local inference.
- Weight Tensor Mapping - Maps compressed tensors to corresponding model layers to ensure hardware compatibility across various precision levels.
- Model Quantization - Reduces model weight precision to decrease the memory footprint and accelerate inference performance.
- Checkpoint Format Transpilations - Provides utilities to convert standard model checkpoints into unified binary structures for portable distribution.
- Model Weight Repair Tools - Includes a tensor fixer to restore missing architectural keys and correct tensor structures in binary model files.
- Transformer Text Encoders - Loads compressed transformer-based text encoders to decrease the total memory required during initialization.
