Mikubill/sd-webui-controlnet
Sd Webui Controlnet
This project is an extension for Stable Diffusion that provides an image-to-image control framework. It serves as a multi-control constraint manager and structural data preprocessor, allowing users to guide the layout and composition of generated images through spatial maps and structural constraints.
The system enables multi-constraint image generation by combining several different control inputs to enforce multiple stylistic or spatial rules within a single generation pass. It provides tools for visual image referencing and precise geometric or anatomical templating to ensure generated images follow a specific visual reference.
The framework covers automated image preprocessing to extract control maps, as well as tuning for guidance strength, timing, and weight balancing between text prompts and spatial references. It also includes capabilities for high-resolution image upscaling to maintain structural consistency.
Programmatic access is provided via API integration for managing control units, executing preprocessors, and triggering generation requests.
Features
- Image Composition Controls - Provides a comprehensive framework for managing spatial layout and composition using external maps and visual references.
- Multi-Control Synthesis - Manages multiple parallel control networks to apply simultaneous spatial and stylistic constraints in a single generation pass.
- Attention Layer Injectors - Modifies internal attention layers during the denoising process to inject precise spatial control signals.
- Control Map Extractors - Extracts structural control maps and annotations from images to prepare them for guidance in diffusion models.
- Diffusion Structural Control - Extracts structural data and creates control maps to guide the spatial layout of AI-generated images.
- Multi-Constraint Orchestration - Enables the combination of several different control inputs to enforce multiple stylistic or spatial rules simultaneously.
- Multi-Constraint Integration - Combines several different control inputs into a single generation to enforce multiple spatial and stylistic constraints.
- Generation Controls - Sets the influence strength and specific generation steps where spatial guidance is applied.
- Precise Guidance Templates - Ensures generated images follow strict geometric or anatomical templates using visual references and preprocessors.
- Guidance Weight Balancers - Adjusts the relative importance between text prompts and control maps to prioritize prompt adherence or structural guidance.
- Guidance Weight Interpolators - Implements dynamic weight blending to balance the influence of text prompts against structural constraints.
- Denoising Step Controllers - Applies control signals only during specific denoising steps to balance structural fidelity and creative freedom.
- Structural Guidance - Provides precise spatial guidance and structural control over the image generation process using specialized models.
- Structural Image Generation - Guides the layout and composition of generated images through the use of spatial maps and structural constraints.
- Structural Map Generators - Transforms input images into simplified structural maps using computer vision models to guide generative networks.
- Visual Reference Guidance - Provides a way to guide image generation using an image as a direct visual reference via attention layer linking.
- Image-to-Image Diffusion Toolkits - Offers a comprehensive toolkit for triggering image-to-image and text-to-image diffusion tasks programmatically.
- Image Super Resolution Models - Maintains structural consistency and detail when scaling images using combined base and high-resolution control passes.
- Resolution Scaling - Generates scaled control images for base and high-resolution passes to ensure structural consistency during upscaling.
- Latent Scaling Mechanisms - Aligns control map resolutions precisely with the latent space dimensions of the generative model.
- Image Preprocessor Registries - Executes specific image preprocessors to generate structural control maps independently of the full generation process.
