Acly/krita-ai-diffusion
Krita Ai Diffusion
This project is a plugin for Krita that integrates Stable Diffusion image generation and editing tools directly into the painting interface. It functions as a remote diffusion backend client, bridging the digital canvas to local or remote servers to handle the computation required for AI image generation.
The system distinguishes itself through a real-time painting interface that translates brushstrokes into generated imagery as the artist works. It acts as a structural orchestrator, using sketches, depth maps, and poses to maintain precise composition, and provides a generative inpainting tool for filling, extending, or modifying specific image regions.
The broader capability surface includes text-to-image generation, image style transfer, and high-resolution tiled upscaling. It supports regional prompting linked to layers, a state-based history tracking system for iterative retrieval, and the integration of node-based workflows for automating complex generation pipelines.
The software includes tools for local AI server hosting and system diagnostics collection to troubleshoot technical integration.
Features
- Generative Painting Assistants - Translates live brushstrokes and sketches into high-quality imagery in real time within the painting interface.
- Interface Plugin Architectures - Integrates generative AI controls and custom panels directly into the Krita painting interface as a plugin.
- AI Image Generation - Functions as an AI image generator that creates artwork based on text prompts and reference images.
- Text-to-Image Generators - Implements pipelines that generate high-resolution artwork from natural language text prompts.
- Image Inpainting - Provides generative inpainting for filling, extending, or modifying specific image regions via masks.
- Image Composition Controls - Orchestrates image generation using sketches, depth maps, and poses to maintain precise structural composition.
- Image Editing - Alters existing image content based on natural language instructions and generative AI.
- Model Server Clients - Provides the connectivity layer that bridges the painting interface to the diffusion backend for heavy computation.
- Structural Guidance - Provides mechanisms for enforcing spatial and geometric constraints using sketches and depth maps during image generation.
- Stable Diffusion Tools - Integrates Stable Diffusion image generation and editing tools directly into the Krita painting interface.
- Remote Infrastructure Clients - Acts as a bridge connecting the digital painting canvas to a local or remote server for AI computation.
- AI Concept Art Workflows - Provides a specialized workflow for creating and refining initial artwork using text prompts and structural guidance.
- Generative Inpainting and Expansion - Adds or removes objects and expands canvas boundaries using generative inpainting and selections.
- Real-Time AI Painting Interfaces - Translates live brushstrokes into generated imagery as the artist works for immediate visual feedback.
- Real-Time Canvas Interpretation - Translates live brushstrokes into generated imagery to provide immediate visual feedback to the artist.
- Generation Iteration Tracking - Stores and previews previous prompts and results to compare different iterations of generated images.
- Diffusion Model Managers - Includes tools for importing, organizing, and switching between different pre-trained diffusion model checkpoints.
- Node-Based Generative Pipelines - Supports node-based generative pipelines that translate visual graphs into sequences of API calls.
- Generative Upscalers - Implements generative upscaling to increase image size and detail to professional resolutions.
- Regional Prompting - Enables the mapping of specific text descriptions to image coordinates using Krita's layer and mask system.
- Neural Style Transfer - Applies styles, compositions, or features from reference images to new generations.
- Generation History Buffers - Caches generation parameters and images in a local database for iterative retrieval and comparison.
- AI Upscaling - Provides high-resolution tiled upscaling to increase image detail without exceeding GPU memory limits.
- Local Generative AI Hosting - Manages and configures diffusion models on local hardware for offline generative processing.
- Asynchronous Task Queues - Implements a background task queue to handle image generation requests without freezing the user interface.
- Diffusion Model Presets - Allows customization of checkpoints, samplers, and weights to create tailored visual styles.
