jina-ai/discoart
Discoart
Discoart is a diffusion model orchestration framework and distributed GPU generation engine designed to automate and scale image generation workflows across hardware clusters. It functions as a generative AI model API, providing HTTP and gRPC endpoints to trigger and retrieve images from diffusion models as a network service.
The system distinguishes itself through a comprehensive task management layer that includes timeline-based prompt and parameter scheduling. It manages the generative art lifecycle by supporting state-based session serialization for recovery, YAML-based configuration management, and the ability to package diffusion model environments for deployment on Kubernetes.
Its capability surface covers text-to-image and image-to-image generation, batch processing, and iterative resolution upscaling. It also includes observability tools for tracking generation metrics and sampling progress via external dashboards and intermediate result polling.
Users can execute tasks and orchestrate workflows through a dedicated command line interface.
Features
- Generative AI APIs - Exposes generative AI capabilities as network services through HTTP and gRPC endpoints for remote clients.
- Image Generation APIs - Provides programmatic interfaces for triggering generative image synthesis from text prompts via a network service.
- Diffusion Model Orchestration - Scales and manages distributed image generation workflows across multiple GPU clusters using containerized diffusion models.
- Distributed GPU Computing - Distributes heavy image processing workloads across multiple hardware accelerators to increase throughput and reduce queue times.
- Generation Parameter Configurations - Enables saving, loading, and exporting generation parameters via YAML files, SVG images, or code.
- Diffusion Models - Integrates third-party or local diffusion model files and configurations to generate images in specific styles.
- Generation Task Managers - Provides a task management layer for scheduling prompts, YAML parameter management, and session recovery.
- Text-to-Image Generators - Produces high-resolution images from natural language text prompts using orchestrated diffusion models.
- GPU Resource Management - Manages CUDA device assignment and replica counts to optimize hardware utilization during generation.
- Model Containerization Tools - Packages diffusion models and dependencies into standardized container images for portable execution across hardware.
- Multi-GPU Workload Distribution - Distributes heavy image generation workloads across multiple GPUs using round-robin scheduling to increase throughput.
- Timeline Scheduling - Implements a timeline-based schema to vary configuration variables and model activation across generation steps.
- Execution Scheduling - Provides precise control over when specific prompts are active, their weights, and the models used across generation steps.
- Orchestration Frameworks - Provides a comprehensive framework for automating and scaling image generation workflows across distributed hardware clusters.
- Prompt Step Schedulers - Controls the activation and weight of prompts and models over a sequence of diffusion steps.
- Serving Workload Scaling - Scales generation capacity by replicating model instances across hardware accelerators to handle higher request volumes.
- Asynchronous Task Queuing - Implements a non-blocking queue system to decouple API requests from intensive GPU processing.
- AI Model API Wrappers - Provides a network service with HTTP and gRPC endpoints for triggering and retrieving images from diffusion models.
- State Serialization - Serializes generation parameters and identifiers to allow recovery of interrupted tasks and previous creative states.
- Networked API Services - Exposes image generation capabilities as a network service via HTTP and gRPC for remote client access.
- Experiment Session Recovery - Allows retrieval of image results and configurations using session IDs to resume work after crashes or timeouts.
- Image-to-Image Diffusion Toolkits - Uses existing images and parameters as initial states for new diffusion generation runs.
- Resolution Upscaling - Increases image resolution and clarity by applying latent diffusion processes to sliding windows.
- Generative Visual Art - Manages the full lifecycle of AI art creation, from prompt scheduling and model configuration to result retrieval.
- Experiment Tracking - Integrates with Weights & Biases to log and analyze image generation experiments on a remote dashboard.
- Model Asset Fetchers - Allows specifying remote URLs to fetch and integrate custom diffusion models for image generation.
- Command Line Interfaces - Provides a dedicated command line tool to execute image generation tasks and orchestrate workflows.
- Model Deployments - Packages diffusion model environments for deployment on Kubernetes clusters to ensure consistent execution across cloud infrastructure.
- Kubernetes Deployments - Exports service configurations into bundles for deployment on Kubernetes cluster-based orchestration systems.
- AI Model Deployers - Packages diffusion model environments into containers for deployment as scalable services on Kubernetes.
- Sliding-Window Inference - Increases image resolution by diffusing overlapping patches to maintain detail at larger scales.
- Multi-Protocol Service Layers - Exposes internal generation workflows through both HTTP and gRPC endpoints to support diverse client communication styles.
- YAML Configuration Files - Utilizes YAML configuration files to define, import, and execute complex image generation tasks.
- Generative Batch Processors - Produces multiple images from the same prompt in a single run to increase throughput and quality.
- Experiment Tracking Dashboards - Streams generation losses and operational health data to remote dashboards for real-time experiment tracking.
