kohya-ss/sd-scripts

Features

• Generative Model Fine-Tuning - Adjusts weights of pre-trained generative models using custom datasets to learn specific subjects, styles, or concepts.
• Custom Stable Diffusion Training - Ships a comprehensive collection of scripts for fine-tuning Stable Diffusion models using various methods and optimizations.
• Diffusion Models - Enables generating images from text or image prompts using trained checkpoints, LoRA weights, and structural guidance.
• Text-to-Image Generators - Provides a pipeline for generating high-resolution images from natural language text prompts.
• Image Inpainting - Provides tools for generative filling of specific image regions using masks and text prompts.
• Image-to-Image Translation - Enables the creation of new images using an existing image as a base with strength control.
• Generative Image Dataset Preparation - Analyzes images to generate descriptive tags and manages resolution buckets and captions for supervised training.
• Image Dataset Preprocessors - Provides utilities for tagging images, generating captions and managing aspect ratio bucketing for model training.
• LoRA Training - Provides a framework for creating and merging low-rank adaptation weights to modify model behavior.
• Model Loading - Loads pre-trained models from remote IDs or local files with precision and VAE options.
• Layer-Wise Learning Rates - Allows freezing specific blocks and assigning individual learning rates to balance training stability.
• Memory Optimization - Reduces VRAM requirements during training and inference through block swapping, mixed precision, and latent caching.
• Layer-Specific Training - Allows selecting specific model blocks or layers for updates to focus training on particular network segments.
• Model Exporting - Exports models and optimizer states with precision settings and automatic checkpoint rotation.
• LoRA-Weighted Image Generations - Generates images using trained LoRA weights with adjustable guidance scales.
• Weight Merging - Integrates trained weights directly into a base model to create a single standalone file.
• Adapter Merging - Combines multiple Low-Rank Adaptation modules into a single model to blend concepts.
• High Resolution Training - Supports training on datasets with multiple image resolutions and adaptive batch sizes for improved flexibility.
• Model Fine-Tuning - A specialized tool for adjusting weights of pre-trained image models through DreamBooth, textual inversion and full fine-tuning.
• Subject-Specific Fine-Tuning - Implements DreamBooth training to specialize generative models on unique subjects using small image sets.
• Textual Inversion Embeddings - Provides tools for optimizing small embedding files to teach new concepts without altering the base model weights.
• Aspect Ratio Bucketing - Optimizes training efficiency by grouping images by resolution and filtering out low-quality small files.
• Model Weight Conversions - Transforms model weights between different architectures and precision formats for compatibility.
• Training Throughput Optimizations - Utilizes specialized precision modes and deep learning primitives to maximize GPU throughput during model training.
• Adapter Module Training - Provides capabilities for training LLLite adapter modules for lightweight conditional image control.
• ControlNet Guidance - Provides capabilities to constrain image structure using hint images and pre-processors.
• Dataset Subset Organization - Organizes images into hierarchical subsets to apply specific repeat counts or caption prefixes.
• Automated Image Tagging - Implements automated image analysis to create descriptive labels in text files for training sets.
• Latent Representation Caching - Speeds up training by storing pre-computed latent representations to disk or memory to avoid redundant calculations.
• Dataset Configurations - Uses configuration files to define training data, high-resolution image settings, and aspect ratio bucketing.
• JIT Kernel Fusion Accelerators - Reduces training step duration by applying just-in-time compilation and kernel fusion on a per-block basis.
• Local Inference - Allows running model inference for image generation directly on local hardware using checkpoints.
• Model Inference - Executes a denoising loop on Flux models to generate images from text prompts.
• Weight Approximation Extraction - Implements the extraction of LoRA weights by calculating the difference between a trained model and its base.
• Text Encoder Adaptation - Allows fine-tuning the text encoder alongside the diffusion model to improve adherence to complex prompts.
• Model Weight Converters - Transforms diffusion model weights between different architectures and precision formats for compatibility.
• LoRA Adapter Loaders - Supports applying and merging multiple LoRA weights to modify the style or content of generated images.
• Model Merging - Provides tools to combine pre-trained weights into a base model to facilitate differential learning.
• Training Caption Processing - Provides tools to add prefixes, suffixes, and shuffle tags to control how the model learns descriptions.
• Image Augmentations - Performs random flips, color adjustments, and face cropping on image subsets to increase training variety.
• In-Training Sample Generation - Generates preview images at specified intervals during training to visually monitor the quality of the model's convergence.
• Training Stability Techniques - Implements noise offsets and input perturbations to ensure consistent convergence and stability during the fine-tuning process.
• Inpainting Training - Provides scripts to train models specifically for filling or replacing regions of images using masks.
• ControlNet Training - Includes specialized training scripts for Flux ControlNet with memory optimizations like CPU offloading.

Star history