Doubiiu/ToonCrafter
ToonCrafter
ToonCrafter is a model that combines latent diffusion, reference-based colorization, and sketch-guided control for cartoon animation and interpolation. It functions as a cartoon video interpolation model, a reference-based colorization model, and a sketch-guided animation tool, all built on a latent diffusion animation framework.
The project distinguishes itself by integrating three core capabilities into a single pipeline: generating smooth intermediate frames between two cartoon images using diffusion-based priors, transferring color and style from a reference image onto black-and-white sketches, and using user-provided sparse sketches as structural guides to direct motion and shape in interpolated video. This is achieved through an architecture that compresses cartoon frames into a lower-dimensional latent space for efficient processing, applies a pretrained diffusion model for frame synthesis, and injects sketch features via cross-attention to steer output.
The system supports sketch-to-color animation, frame interpolation, and sketch-guided generation, enabling users to apply a reference palette to a sketch, generate smooth cartoon frames, and steer interpolation with sketches. The documentation and install surface provide the necessary resources to run the model locally.
