# nunchaku-ai/comfyui-nunchaku

**Attribution required: if you use, quote, or summarise this content, you must credit and link back to [awesome-repositories.com](https://awesome-repositories.com/repository/nunchaku-ai-comfyui-nunchaku).**

2,778 stars · 148 forks · Python · apache-2.0

## Links

- GitHub: https://github.com/nunchaku-ai/ComfyUI-nunchaku
- Homepage: https://nunchaku.tech/docs/ComfyUI-nunchaku/
- awesome-repositories: https://awesome-repositories.com/repository/nunchaku-ai-comfyui-nunchaku.md

## Topics

`comfyui` `diffusion` `flux` `genai` `mlsys` `quantization`

## Description

ComfyUI-nunchaku is a 4-bit diffusion inference engine and a set of nodes for running low-precision quantized diffusion models within ComfyUI visual workflows. It provides a backend that reduces memory overhead and increases generation speed for transformer models.

The project includes specialized tools for identity-preserving generation and an image-to-image guidance toolkit that uses depth maps and reference images. It also features a multimodal visual question answering implementation and a utility for merging multiple quantized model files into single unified files.

The engine covers a broad range of image generation and editing capabilities, including text-to-image generation, structural image control, text-based inpainting, and face restoration. It implements various performance optimizations such as device-aware memory offloading, fused kernel projections, and joint image-text attention.

The system includes hardware-aware dependency installation that detects system hardware to deploy compatible pre-compiled binaries.

## Tags

### Artificial Intelligence & ML

- [Text-to-Image Generators](https://awesome-repositories.com/f/artificial-intelligence-ml/generative-ai-resources/diffusion-visual-models/generative-ai-pipelines/text-to-image-generators.md) — Produces visual assets from written descriptions using a memory-efficient, 4-bit quantized inference engine. ([source](https://nunchaku.tech/docs/ComfyUI-nunchaku/workflows/flux.html))
- [Stable Diffusion Inference Engines](https://awesome-repositories.com/f/artificial-intelligence-ml/stable-diffusion-inference-engines.md) — Provides a high-performance 4-bit inference engine specifically for executing transformer-based diffusion models.
- [Identity Adapters](https://awesome-repositories.com/f/artificial-intelligence-ml/generative-ai-resources/diffusion-visual-models/generative-ai-models/diffusion-models/identity-adapters.md) — Integrates specialized identity adapters into diffusion models to maintain consistent person-specific features. ([source](https://nunchaku.tech/docs/ComfyUI-nunchaku/api/toc.html))
- [Image-to-Image Diffusion Toolkits](https://awesome-repositories.com/f/artificial-intelligence-ml/generative-ai-resources/diffusion-visual-models/generative-ai-pipelines/text-to-image-generators/image-inpainting/image-to-image-diffusion-toolkits.md) — Ships a toolkit for controlling image generation using depth maps, reference images, and identity weights.
- [Node-Based Generative Pipelines](https://awesome-repositories.com/f/artificial-intelligence-ml/generative-ai-resources/workflow-execution-backends/node-based-generative-pipelines.md) — Integrates low-precision model execution into visual graph-based workflows via ComfyUI nodes.
- [Identity-Driven Image Generation](https://awesome-repositories.com/f/artificial-intelligence-ml/identity-driven-image-generation.md) — Extracts identity embeddings from a reference image to keep person-specific features consistent during generation. ([source](https://nunchaku.tech/docs/ComfyUI-nunchaku/workflows/pulid.html))
- [Joint Attention Mechanisms](https://awesome-repositories.com/f/artificial-intelligence-ml/joint-attention-mechanisms.md) — Computes joint attention for image and text streams to accelerate inference speed. ([source](https://nunchaku.tech/docs/ComfyUI-nunchaku/api/models.qwenimage.html))
- [LoRA Adapter Loaders](https://awesome-repositories.com/f/artificial-intelligence-ml/model-weight-management/lora-adapter-loaders.md) — Integrates LoRA weights with adjustable strengths to modify model output characteristics. ([source](https://nunchaku.tech/docs/ComfyUI-nunchaku/api/nodes.html))
- [Quantized Inference Accelerators](https://awesome-repositories.com/f/artificial-intelligence-ml/quantized-inference-runtimes/quantized-inference-accelerators.md) — Performs efficient image-based inference using low-precision neural networks within a workflow. ([source](https://nunchaku.tech/docs/ComfyUI-nunchaku/api/model_base.qwenimage.html))
- [Quantized Text-to-Image Generation](https://awesome-repositories.com/f/artificial-intelligence-ml/quantized-text-to-image-generation.md) — Provides a memory-efficient 4-bit inference engine for generating images from text prompts. ([source](https://nunchaku.tech/docs/ComfyUI-nunchaku/workflows/toc.html))
- [Joint Image-Text Attention](https://awesome-repositories.com/f/artificial-intelligence-ml/attention-mechanisms/hybrid-attention/joint-image-text-attention.md) — Accelerates inference by computing combined attention for image and text streams within a single operation.
- [Image-to-Image Translation](https://awesome-repositories.com/f/artificial-intelligence-ml/generative-ai-resources/diffusion-visual-models/generative-ai-pipelines/text-to-image-generators/image-inpainting/image-to-image-translation.md) — Converts an existing image into a new visual output using a reference source and text prompt. ([source](https://demo.nunchaku.tech/))
- [Transformer Projection Kernels](https://awesome-repositories.com/f/artificial-intelligence-ml/gpu-kernel-implementations/kernel-composition-frameworks/fused-gpu-kernel-composition/transformer-projection-kernels.md) — Accelerates inference by combining projections and rotations into single optimized kernels within transformer layers.
- [Hardware Device Management](https://awesome-repositories.com/f/artificial-intelligence-ml/hardware-device-management.md) — Automatically manages the movement of model weights between CPU and GPU hardware. ([source](https://nunchaku.tech/docs/ComfyUI-nunchaku/api/mixins.model.html))
- [Composition-Controlled Generators](https://awesome-repositories.com/f/artificial-intelligence-ml/image-generation/composition-controlled-generators.md) — Directs the image generation process based on spatial or structural input to ensure precise composition. ([source](https://nunchaku.tech/docs/ComfyUI-nunchaku/workflows/zimage.html))
- [Image Editing](https://awesome-repositories.com/f/artificial-intelligence-ml/image-generation/image-editing.md) — Provides tools for modifying existing visual content using generative AI instructions. ([source](https://nunchaku.tech/docs/ComfyUI-nunchaku/workflows/qwenimage.html))
- [Text-Guided Inpainting](https://awesome-repositories.com/f/artificial-intelligence-ml/image-generation/image-editing/generative-masking/text-guided-inpainting.md) — Fills or modifies specific image regions using text descriptions and low-precision models. ([source](https://nunchaku.tech/docs/ComfyUI-nunchaku/workflows/fill.html))
- [Quantized Editing](https://awesome-repositories.com/f/artificial-intelligence-ml/image-generation/image-editing/quantized-editing.md) — Modifies existing images using compressed neural networks and acceleration techniques to reduce memory usage. ([source](https://nunchaku.tech/docs/ComfyUI-nunchaku/workflows/kontext.html))
- [Reference-Conditioned Generation](https://awesome-repositories.com/f/artificial-intelligence-ml/image-generation/reference-conditioned-generation.md) — Creates new images from a reference source utilizing a vision-language model. ([source](https://nunchaku.tech/docs/ComfyUI-nunchaku/workflows/redux.html))
- [Style Transfers](https://awesome-repositories.com/f/artificial-intelligence-ml/image-generation/style-transfers.md) — Generates new images based on an input reference while maintaining structural consistency via depth detection. ([source](https://nunchaku.tech/docs/ComfyUI-nunchaku/workflows/depth.html))
- [Inference Optimization Kernels](https://awesome-repositories.com/f/artificial-intelligence-ml/inference-optimization-kernels.md) — Implements fused kernel projections and rotations to accelerate transformer model inference speed. ([source](https://nunchaku.tech/docs/ComfyUI-nunchaku/api/models.zimage.html))
- [Weight Merging Utilities](https://awesome-repositories.com/f/artificial-intelligence-ml/machine-learning/architectures/instruction-tuned-language-models/weight-space-merging-techniques/weight-merging-utilities.md) — Merges multiple quantized weight files into a single state to blend styles or preserve identities.
- [Local Inference Engines](https://awesome-repositories.com/f/artificial-intelligence-ml/machine-learning/infrastructure/model-optimization-and-inference/serving-and-runtime/large-language-model-optimization/local-inference-engines/local-inference-engines.md) — Manages the installation and removal of the optimized 4-bit inference engine. ([source](https://nunchaku.tech/docs/ComfyUI-nunchaku/api/nodes.tools.installers.html))
- [Weight Offloading](https://awesome-repositories.com/f/artificial-intelligence-ml/model-weight-management/weight-offloading.md) — Implements dynamic movement of model weights between CPU and GPU memory to support high-resolution generation.
- [Model Weight Utilities](https://awesome-repositories.com/f/artificial-intelligence-ml/model-weight-utilities.md) — Combines multi-file quantized model directories into single unified files for streamlined distribution.
- [Multimodal Model Runners](https://awesome-repositories.com/f/artificial-intelligence-ml/multimodal-models/multimodal-model-runners.md) — Implements a visual question answering node that processes images and text using quantized multimodal models.
- [Model Merging](https://awesome-repositories.com/f/artificial-intelligence-ml/pre-trained-model-application/model-merging.md) — Combines multiple pre-trained diffusion model weights to blend styles or capabilities. ([source](https://nunchaku.tech/docs/ComfyUI-nunchaku/nodes/toc.html))
- [Quantized Model Consolidation](https://awesome-repositories.com/f/artificial-intelligence-ml/pre-trained-model-application/model-merging/quantized-model-consolidation.md) — Combines multiple quantized model files into a single unified file for easier distribution. ([source](https://nunchaku.tech/docs/ComfyUI-nunchaku/nodes/tools.html))
- [Weight Quantization](https://awesome-repositories.com/f/artificial-intelligence-ml/quantized-inference-runtimes/weight-quantization.md) — Compresses and combines model weights into lower-precision formats to reduce memory footprint. ([source](https://nunchaku.tech/docs/ComfyUI-nunchaku/api/nodes.tools.html))
- [Quantized Model Implementations](https://awesome-repositories.com/f/artificial-intelligence-ml/quantized-inference-runtimes/weight-quantization/quantized-model-implementations.md) — Implements specialized low-precision transformer model versions to reduce memory usage within visual workflows. ([source](https://nunchaku.tech/docs/ComfyUI-nunchaku/api/wrappers.flux.html))
- [Visual Reference Prompting](https://awesome-repositories.com/f/artificial-intelligence-ml/reasoning-models/reasoning-pipelines/visual-prompt-enhancers/visual-reference-prompting.md) — Influences generated output by injecting visual reference data into the model pipeline via specialized adapters. ([source](https://nunchaku.tech/docs/ComfyUI-nunchaku/nodes/ipadapter.html))
- [Visual Question Answering](https://awesome-repositories.com/f/artificial-intelligence-ml/visual-question-answering.md) — Provides a visual question answering implementation that processes images and text using quantized multimodal models. ([source](https://nunchaku.tech/docs/ComfyUI-nunchaku/workflows/toc.html))

### Part of an Awesome List

- [Quantized Inference Runtimes](https://awesome-repositories.com/f/awesome-lists/ai/local-llm-execution/quantized-inference-runtimes.md) — Executes low-precision neural networks to reduce memory usage and increase image generation speed. ([source](https://nunchaku.tech/docs/ComfyUI-nunchaku/_sources/index.rst))

### Development Tools & Productivity

- [ComfyUI Custom Node Suites](https://awesome-repositories.com/f/development-tools-productivity/comfyui-custom-node-suites.md) — Implements a suite of ComfyUI custom nodes for running quantized diffusion models in visual workflows.

### Software Engineering & Architecture

- [Low-Bit Inference Engines](https://awesome-repositories.com/f/software-engineering-architecture/memory-layout-optimizations/bit-packed-storage/low-bit-inference-engines.md) — Provides an execution engine optimized for running models with 4-bit packed weights to reduce memory overhead.

### Operating Systems & Systems Programming

- [Model Memory Managers](https://awesome-repositories.com/f/operating-systems-systems-programming/kernel-core-internals/process-and-memory-management/memory-management/allocation-strategies/dynamic-memory-allocation/custom-memory-allocators/managed-memory-allocators/model-memory-managers.md) — Controls the allocation and offloading of model weights between system memory and GPU to optimize VRAM. ([source](https://nunchaku.tech/docs/ComfyUI-nunchaku/api/mixins.html))