Awesome GitHub RepositoriesDiffusion Pipelines

ComfyUI is a modular generative AI workflow orchestrator and node-based GUI for designing and executing complex diffusion model pipelines. It functions as both a visual interface for building generative logic graphs and a programmable backend API that exposes diffusion model operations for external integration. The system distinguishes itself through a graph-based execution model that supports differential workflow execution, re-running only modified nodes to reduce computation. It features dynamic model offloading to manage memory between system RAM and GPU VRAM and utilizes metadata-embedde

Fooocus is a generative image interface designed to simplify the creation of high-quality visual content from text descriptions. It functions as a latent diffusion pipeline and model orchestrator, managing the complex interactions between neural network layers, mathematical samplers, and hardware resource allocation to produce professional-grade imagery. The project distinguishes itself through a sophisticated prompt engineering engine and modular style management. Users can dynamically modify output characteristics by injecting style adapters directly into prompts or by utilizing wildcards a

Diffusers is a PyTorch-based library and generative AI framework used to build, train, and deploy diffusion pipelines for producing multi-modal media. It provides a suite of tools for generating images, video, and audio from natural language descriptions, as well as specialized systems for text-to-image generation. The project differentiates itself through a modular architecture that separates noise schedulers, pretrained model blocks, and pipeline compositions. This structure allows for the construction of custom generation workflows and the ability to swap individual components of the diffu

ComfyUI-Manager is an integrated utility designed to manage software dependencies, custom extensions, and machine learning assets within generative image workflow environments. It provides a centralized interface for the installation, update, and organization of third-party nodes and model files, ensuring that all necessary resources are available for processing tasks. The project distinguishes itself by automating the lifecycle of these components through both a graphical user interface and command-line utilities. It handles complex environment maintenance by fetching remote repositories, tr

DiffSynth-Studio is a comprehensive platform for the lifecycle management of generative diffusion models, providing a unified environment for inference, fine-tuning, and training. It utilizes a modular pipeline architecture and a standardized abstraction layer to support consistent workflows across diverse model configurations for image and video generation. The platform distinguishes itself through a memory-optimized inference engine that dynamically manages resources to facilitate high-resolution generation on constrained hardware. It also integrates specialized training capabilities, inclu

This is a PyTorch-based implementation of diffusion models for synthesizing photorealistic images and video. It provides a framework for text-to-image and text-to-video generation, as well as unconditional image synthesis. The system utilizes a cascading diffusion pipeline to produce high-resolution imagery by passing low-resolution outputs through a sequence of super-resolution models. It also includes capabilities for image inpainting, allowing the reconstruction of masked or missing regions of visual media guided by surrounding context and text prompts. The project includes tools for diff

IF is a text-to-image diffusion system that translates natural language descriptions into visual imagery. The project provides a generative pipeline for creating images, an inpainting tool for modifying specific image sections, and a super-resolution upscaler to increase pixel density and clarity. The system includes a concept fine-tuning framework that allows for the teaching of new visual concepts by updating a small set of parameters. It also supports image style transfer to apply the aesthetic characteristics of a reference image to a new output.

mmagic is a multimodal training pipeline and framework for generative AI, focusing on visual synthesis and restoration. It provides the infrastructure to build and train models for tasks such as text-to-image and text-to-video generation, 3D-aware content synthesis, and high-fidelity image translation using diffusion models and generative adversarial networks. The project distinguishes itself through specialized capabilities for generative model personalization, including techniques for fine-tuning subjects and styles. It also supports advanced visual manipulations such as latent space interp

This is a classifier-guided diffusion framework for high-fidelity image generation. It implements a cascaded diffusion pipeline that chains a base diffusion model with a dedicated upsampler to progressively increase image resolution in stages, and uses classifier-guided diffusion sampling to steer the reverse diffusion process toward higher-quality outputs. The framework provides tools for training diffusion models from scratch using distributed processes with gradient accumulation, as well as training classifier models that provide gradient-based guidance during sampling. It supports both un

StableCascade is a generative AI system and latent diffusion framework designed for text-to-image synthesis and image-to-image transformations. It utilizes a multi-stage cascade architecture that encodes and decodes images via a latent space to produce high-fidelity visual imagery. The system includes a cascade diffusion pipeline for controlling image structure through inpainting, outpainting, and super-resolution. It also provides a toolkit for image-to-image generation and the creation of image variations using embeddings. The framework supports model optimization through low-rank adaptati

This project provides a comprehensive technical guide and framework for engineering large-scale machine learning systems. It covers the full lifecycle of model development, focusing on the infrastructure and computational principles required to build, train, and serve generative AI models across distributed GPU clusters. The repository distinguishes itself by offering deep-dive tutorials and implementation strategies for complex system challenges. It emphasizes high-performance architectural primitives, such as collective communication orchestration, distributed tensor sharding, and static gr

Videocrafter est un modèle de diffusion latente conçu pour la synthèse vidéo par IA. Il fonctionne à la fois comme un système de génération texte-vers-vidéo et image-vers-vidéo, synthétisant des séquences vidéo de haute qualité à partir de prompts textuels descriptifs ou d'entrées d'images statiques. Le modèle utilise un réseau de neurones basé sur la diffusion pour transformer les entrées en contenu animé, assurant la cohérence visuelle et la cohérence temporelle à travers les séquences générées. Cela permet la création de clips vidéo personnalisés et l'animation d'images statiques en mouvement fluide.

Multimodal is a machine learning library built on PyTorch for training large-scale models that combine text, image, audio, and video data streams. It functions as a deep learning framework dedicated to generative diffusion models, multi-task training, and vision-language tasks. The library supplies modular building blocks, discrete latent codebook quantization, shared-space embeddings, and stackable adapter layers to handle diverse conditional inputs during training and inference. The framework supports specific architectures for diffusion models, text-to-video generation, image-text retrieva

Ce projet est un framework logiciel intégré conçu pour faciliter la synthèse d'images génératives et l'inférence de modèles haute performance sur les processeurs et le matériel graphique Intel. Il fournit un moteur d'inférence spécialisé qui exécute des modèles de diffusion latente pour transformer des descriptions en langage naturel en sorties visuelles. La bibliothèque se distingue en tirant parti de la boîte à outils OpenVINO pour optimiser les modèles de machine learning pour des architectures matérielles Intel spécifiques. En utilisant l'accélération matérielle au niveau du noyau et l'optimisation de graphe statique, le framework améliore le débit d'exécution et l'efficacité des ressources lors de l'exécution de pipelines de réseaux de neurones complexes. Le logiciel prend en charge une gamme de capacités génératives, y compris la génération texte-vers-image, la transformation image-vers-image et l'inpainting d'image sélectif. Ces fonctionnalités permettent aux utilisateurs de créer de nouveaux contenus visuels ou de modifier des images existantes tout en conservant la composition et la structure originales grâce à des processus de diffusion accélérés par le matériel.