lllyasviel/IC-Light
IC Light
IC-Light is a diffusion-based image editor and generative tool designed for controlling the illumination of foreground subjects. It functions as an image relighting system that uses latent diffusion models to modify lighting effects on isolated subjects.
The project provides two primary methods for lighting control: text-based relighting, which uses descriptive prompts and lighting directions, and background-based relighting, which conditions the foreground lighting to match the visual properties of a provided background image.
Beyond illumination, the system includes a surface normal estimator that derives geometric surface information by analyzing lighting variations across multiple processed images. These capabilities support the production of visual effects and consistent lighting for digital art and composites.
Features
- Image Diffusion Models - Employs image diffusion models to synthesize lighting and color details while maintaining the original image structure.
- AI Image Relighting - Changes the lighting of a foreground subject using text prompts or a background image.
- Generative Lighting Control - Implements generative lighting control to adjust atmospheric conditions and illumination on isolated image subjects.
- Relighting Models - Generates modified illumination for foreground subjects using text prompts or background image conditioning.
- Reference-Conditioned Generation - Provides reference-conditioned generation to match foreground lighting with the visual properties of a background image.
- Diffusion-Based Editors - Provides a diffusion-based editor for creating consistent lighting effects on isolated subjects.
- Image Relighting - Imposes consistent lighting on foreground subjects by analyzing the visual properties of a provided background image.
- Static Image Relighting - Modifies foreground illumination using text prompts and lighting directions to create consistent lighting effects.
- Cross-Attention Mechanisms - Uses cross-attention mechanisms to map textual lighting descriptions to specific spatial regions of the foreground subject.
- Latent Space Manipulations - Modifies compressed latent representations of images to change illumination before decoding them back into pixels.
- Iterative Denoising Pipelines - Implements an iterative denoising pipeline to refine relit images toward a consistent lighting state.
- Surface Normal Estimation - Estimates surface normals by analyzing lighting variations across multiple relighting outputs to determine depth and orientation.
- AI Lighting Compositing - Creates consistent lighting effects on image subjects for use in digital art or composite photography.
