# ant-research/magicquill

**Attribution required: if you use, quote, or summarise this content, you must credit and link back to [awesome-repositories.com](https://awesome-repositories.com/repository/ant-research-magicquill).**

_How this analysis was created: the description and tags below were written by an AI model that read this project's README and public documentation pages; stars, license and language come straight from the GitHub API. The model does not read the source code._

3,682 stars · 390 forks · Python · NOASSERTION

## Links

- GitHub: https://github.com/ant-research/MagicQuill
- Homepage: https://magicquill.art
- awesome-repositories: https://awesome-repositories.com/repository/ant-research-magicquill.md

## Topics

`aigc` `gradio` `image-editing` `mllm`

## Description

MagicQuill is a suite of interactive tools for image segmentation, diffusion-based editing, layered composition, and prompt-guided visual synthesis. It functions as a diffusion model image editor and a layered visual composition tool, enabling the addition, removal, and recoloring of image elements through a combination of sketches and text prompts.

The system features a prompt-guided image generator that predicts editing instructions by analyzing user drawings to automatically populate text prompts. It allows for visual style control by swapping generative model weights to shift outputs between realistic, fantasy, or anime aesthetics.

The toolset covers a broad range of capabilities including object segmentation for asset extraction, spatial-guidance control for geometry and color mapping, and guided inpainting. It provides a canvas for arranging extracted props and masks to structure complex scenes before final image synthesis.

## Tags

### Artificial Intelligence & ML

- [Image Editing](https://awesome-repositories.com/f/artificial-intelligence-ml/image-generation/image-editing.md) — Provides an interactive system for adding, removing, and recoloring image elements using brushes and text prompts.
- [Sketch-Based Image Geometry Control](https://awesome-repositories.com/f/artificial-intelligence-ml/ai-video-generators/video-motion-controllers/sketch-based-image-geometry-control.md) — Defines the edges of an object using brushes to control the physical geometry of the final image output. ([source](https://magicquill.art/v2/))
- [Image Diffusion Models](https://awesome-repositories.com/f/artificial-intelligence-ml/computer-vision-systems/image-diffusion-models.md) — Implements a generative engine that refines noise into images using diffusion models.
- [Image Segmentation](https://awesome-repositories.com/f/artificial-intelligence-ml/computer-vision-systems/image-segmentation.md) — Isolates foreground objects using points and bounding boxes to create reusable visual assets.
- [Diffusion Models](https://awesome-repositories.com/f/artificial-intelligence-ml/generative-ai-resources/diffusion-visual-models/generative-ai-models/diffusion-models.md) — Implements an interactive system for performing local modifications using diffusion architectures.
- [Editing Prompt Generation](https://awesome-repositories.com/f/artificial-intelligence-ml/image-generation/image-editing/editing-prompt-generation.md) — Predicts editing instructions by analyzing user drawings to automatically populate text prompts for image synthesis.
- [Element Removal](https://awesome-repositories.com/f/artificial-intelligence-ml/image-generation/image-editing/generative-masking/prompt-based-masking/element-removal.md) — Enables deleting specific details or redrawing areas of an image using text-based prompts. ([source](https://cdn.jsdelivr.net/gh/ant-research/magicquill@main/README.md))
- [Regional Modifications](https://awesome-repositories.com/f/artificial-intelligence-ml/image-region-reconstruction/regional-modifications.md) — Provides a targeted interface to paint specific regions of an image to restrict modifications to precise areas. ([source](https://magicquill.art))
- [SAM-Based Implementations](https://awesome-repositories.com/f/artificial-intelligence-ml/neural-networks/image-segmentations/sam-based-implementations.md) — Isolates foreground assets using point and box prompts via the Segment Anything Model architecture.
- [Prompt-Guided Element Insertion](https://awesome-repositories.com/f/artificial-intelligence-ml/prompt-guided-element-insertion.md) — Inserts new details or objects into a picture by applying prompt-guided brush strokes to specific areas. ([source](https://cdn.jsdelivr.net/gh/ant-research/magicquill@main/README.md))
- [Model Checkpoint Swapping](https://awesome-repositories.com/f/artificial-intelligence-ml/ai-model-integrations/dynamic-model-swapping/model-checkpoint-swapping.md) — Swaps generative model weights between different checkpoints to shift outputs between realistic, fantasy, or anime styles.
- [Model Weight Style Switching](https://awesome-repositories.com/f/artificial-intelligence-ml/model-weight-style-switching.md) — Shifts visual output between realistic, fantasy, or anime aesthetics by swapping generative model weights. ([source](https://cdn.jsdelivr.net/gh/ant-research/magicquill@main/README.md))
- [Multimodal Image Composition](https://awesome-repositories.com/f/artificial-intelligence-ml/multimodal-image-composition.md) — Combines hand-drawn sketches, color maps, and text descriptions to build complex visual scenes.

### Part of an Awesome List

- [Image Editing and Manipulation](https://awesome-repositories.com/f/awesome-lists/ai/image-editing-and-manipulation.md) — Allows adding, removing, and recoloring image elements through a combination of sketches and text prompts.
- [Spatial Guidance Controls](https://awesome-repositories.com/f/awesome-lists/ai/image-editing-and-manipulation/spatial-guidance-controls.md) — Directs the geometry and color of generated elements using sketch-based edges and color maps.
- [Prompt-Based Segmentations](https://awesome-repositories.com/f/awesome-lists/ai/image-segmentation/prompt-based-segmentations.md) — Isolates image regions using points, bounding boxes, or erasers to create precise foreground assets. ([source](https://magicquill.art/v2/))

### Graphics & Multimedia

- [Mask-Guided Image Editors](https://awesome-repositories.com/f/graphics-multimedia/ai-image-masking/mask-guided-image-editors.md) — Restricts generative updates to specific image coordinates using additive and subtractive brush strokes.
- [Generative Scene Assembly](https://awesome-repositories.com/f/graphics-multimedia/canvas-image-positioning/canvas-compositing/generative-scene-assembly.md) — Combines extracted foreground props and spatial masks on a virtual canvas to guide the generation of complex scenes.
- [Generative Inpainting and Expansion](https://awesome-repositories.com/f/graphics-multimedia/generative-inpainting-and-expansion.md) — Fills or replaces regions of an image using sketches and colors to control generated content.
- [Generative Scene Composition](https://awesome-repositories.com/f/graphics-multimedia/generative-scene-composition.md) — Combines extracted foreground props and target areas on a canvas to build complex visual scenes. ([source](https://magicquill.art/v2))
- [Generative Shape Guidance](https://awesome-repositories.com/f/graphics-multimedia/generative-shape-guidance.md) — Allows users to sketch or mask edges using additive and subtractive brushes to control generated shapes. ([source](https://magicquill.art/v2))
- [Generative Layout Composition](https://awesome-repositories.com/f/graphics-multimedia/layered-visual-composition/generative-layout-composition.md) — Provides a canvas for arranging extracted props and masks to build complex scenes before final generation.
- [Object Removal](https://awesome-repositories.com/f/graphics-multimedia/temporal-video-background-removal/object-removal.md) — Uses a subtractive brush to precisely erase objects from a scene while maintaining the background. ([source](https://magicquill.art/v2))
- [Visual Asset Extractions](https://awesome-repositories.com/f/graphics-multimedia/visual-asset-extractions.md) — Isolates specific objects from an image through segmentation to create foreground props for use in new compositions.
- [Predictive Prompt Completion](https://awesome-repositories.com/f/graphics-multimedia/3d-geometry-editing/prompt-based-geometry-editing/predictive-prompt-completion.md) — Predicts intended objects or actions from user drawings to automatically populate text-based editing prompts. ([source](https://cdn.jsdelivr.net/gh/ant-research/magicquill@main/README.md))
- [Generative Color Mapping](https://awesome-repositories.com/f/graphics-multimedia/generative-color-mapping.md) — Allows filling specific image areas with selected brush colors to match a target color palette during generation. ([source](https://cdn.jsdelivr.net/gh/ant-research/magicquill@main/README.md))
- [Generative Color Overlays](https://awesome-repositories.com/f/graphics-multimedia/generative-color-overlays.md) — Overlays semi-transparent colors on specific areas to dictate the color scheme of the generated result. ([source](https://magicquill.art/v2/))
- [Generative Object Compositions](https://awesome-repositories.com/f/graphics-multimedia/image-composition/generative-object-compositions.md) — Places predefined foreground assets onto a canvas to guide the generation of specific objects or characters. ([source](https://magicquill.art/v2/))
- [Localized Generative Edits](https://awesome-repositories.com/f/graphics-multimedia/image-manipulation-tools/localized-generative-edits.md) — Applies precise generative changes to small areas of a photo while keeping the rest of the image unchanged.