Open-source alternatives to DiffBIR

Zongsheng Yue, Chen Change Loy

RestorePhotos is an AI face restoration tool and deep learning image upscaler designed to remove blur and reconstruct lost details in degraded facial photographs. It functions as a face photo enhancer and a generative adversarial network image processor that transforms low-quality pixels into high-resolution facial features. The system utilizes a GPU-accelerated inference engine to run machine learning models for real-time image restoration. This hardware acceleration supports the heavy matrix multiplications and tensor-based operations required to sharpen facial images and improve visual fid

Real-ESRGAN is a deep learning restoration pipeline designed to enhance low-resolution media and improve the visual quality of damaged photographs. It functions as a generative image upscaler that reconstructs high-resolution details from source inputs by utilizing neural networks trained to fill in missing information and remove noise. The project distinguishes itself as a blind super-resolution tool, meaning it improves image sharpness and fidelity without requiring prior knowledge of the specific degradation applied to the source. It employs high-order degradation modeling to address compl

Stable Diffusion Web UI is a browser-based interface for generating, editing, and upscaling images and videos using latent diffusion models. It functions as a text-to-image generator, an AI image editor, and a tool for increasing image resolution and clarity. The system includes capabilities for custom model training, specifically allowing the creation of textual inversion embeddings to teach a model new concepts and visual styles from user photos. It also provides tools for AI video production, generating short clips from text prompts. The software covers image-to-image transformation, imag

BasicSR is a PyTorch-based image restoration toolbox and framework designed for training and deploying deep learning models to upscale, denoise, and deblur images and videos. It serves as a comprehensive system for image super-resolution and video quality restoration, providing the necessary infrastructure to recover fine visual details and increase pixel density. The project distinguishes itself through specialized toolkits for facial image enhancement and high-fidelity face synthesis, as well as a dedicated video quality restoration suite that utilizes deformable convolutions and generative

QualityScaler is an AI video upscaler and local media processing tool designed to increase the resolution and visual quality of videos and images. It uses deep learning models to enhance detail and remove noise, operating as an offline application that executes all computations on local hardware. The project functions as a GPU-accelerated media processor that distributes workloads across multiple graphics cards to increase rendering speed. To prevent memory overflow during high-resolution tasks, it employs a tiled image processing method that splits large assets into smaller sections. The sy

This project is a neural network extension for Stable Diffusion that provides spatial control and geometric consistency for text-to-image generation. It functions as an image structure controller and conditioning tool, enabling the use of external inputs to guide the layout and geometry of generated imagery. The framework is distinguished by its ability to transform input images into structural guides through various preprocessors. These include the extraction of depth maps, normal maps, and human pose landmarks, as well as the detection of Canny edges, anime lineart, and straight architectur

This project is an AI upscaling framework and deep learning image restorer designed to estimate original source pixels from low-resolution inputs. It functions as a super-resolution reconstruction system that transforms pixelated images into high-resolution versions by restoring high-frequency details and sharpening edges. The system utilizes a convolutional neural network pipeline to analyze pixel data and perform digital image restoration. It employs pixel-shuffle upsampling to rearrange channel dimensions into spatial dimensions, which increases resolution while reducing checkerboard artif

This project is an unsupervised image restoration tool that uses a convolutional neural network as a structural prior to reconstruct images from noisy or incomplete data. It functions as a neural network image prior, utilizing the inherent biases of the network architecture to restore pixels without the need for a pre-trained dataset or external learning. The system performs zero-shot image restoration by treating the network architecture itself as a regularization term. It uses a randomly initialized encoder-decoder structure and iterative gradient descent to minimize pixel-wise loss, recove

Lama is an image restoration framework and deep learning model designed for image inpainting and object removal. It provides the tools necessary to train and evaluate neural networks that fill masked areas and repair corrupted visual data. The system utilizes a Fourier convolution neural network to maintain global image structure and reconstruct periodic patterns. This architecture allows for resolution-independent inference, enabling the processing of high-resolution images without increasing memory or computational requirements. The project includes a synthetic dataset generator that creat

This is the code repository of the following paper to train and perform inference with patch-based diffusion models for image restoration under adverse weather conditions.

Yinhuai Wang, Jiwen Yu, Jian Zhang Peking University and PCL \*denotes equal contribution

Zongsheng Yue, Jianyi Wang, Chen Change Loy

Yuanzhi Zhu, Kai Zhang, Jingyun Liang, Jiezhang Cao, Bihan Wen, Radu Timofte, Luc Van Gool.

Ziwei Luo, Fredrik K. Gustafsson, Zheng Zhao, Jens Sjölund, Thomas B. Schön Department of Information Technology, Uppsala University

The repository contains reproducible PyTorch source code of our paper Solving Linear Inverse Problems Provably via Posterior Sampling with Latent Diffusion Models. We present the first framework to solve general inverse problems leveraging pre-trained latent diffusion models. Previously proposed…

CVPR 2024: Residual Denoising Diffusion Models

NeurIPS 2023 PGDiff: Guiding Diffusion Models for Versatile Face Restoration via Partial Guidance

SUPIR is an AI image upscaler and restoration system designed to remove artifacts and restore quality to real-world photographs. It functions as a diffusion-based image enhancer and restoration tool that uses large-scale model scaling to produce high-resolution results with photorealistic details. The system balances visual aesthetics with input fidelity, allowing for a trade-off between strict adherence to the original image and the overall visual appeal of the output. It leverages large-scale model inference to improve image clarity and maintain realistic details during the upscaling proces

Pulse is a face image super-resolution tool and self-supervised image enhancer. It functions as a generative model image upsampler and latent space optimization tool designed to increase photo resolution and recover image details. The system differentiates itself by using latent space exploration and spherical constraints to find high-fidelity matches within a generative model. It employs geodesic distance measurement and spherical latent space optimization to regularize representations and maintain parameter radii during the recovery process. The project covers facial image restoration thro

This project is a deep learning library built for single-image super-resolution and visual enhancement. It provides a framework for training and deploying neural network architectures designed to reconstruct high-resolution images from low-resolution sources, effectively recovering fine details and removing artifacts caused by downscaling or compression. The library distinguishes itself through the implementation of generative adversarial networks and residual block architectures, which work together to improve the realism and clarity of upscaled outputs. It supports training through both pix

ESRGAN is a deep learning image restoration framework designed for image super-resolution. It uses a generative adversarial network system to upscale low-resolution images into high-quality versions with sharp visual details and recovered fine textures. The framework implements a perceptual super-resolution model that optimizes the trade-off between perceived visual quality and pixel-level signal-to-noise ratio. It includes weight-interpolation blending to allow for the adjustment of visual sharpness and signal-to-noise ratios by mixing weights from different trained models. The system cover

Hallo2 is an AI video generation tool and audio-driven portrait animation framework designed to transform static images into speaking videos. It functions as a portrait image animator that synchronizes a single photo with an audio track to produce high-resolution talking head videos. The system includes a distributed animation trainer for fine-tuning deep learning models using custom datasets and distributed computing resources. It employs hierarchical video generation and temporal consistency modeling to produce long-form character animations that remain stable over extended durations. The

This project is an AI image upscaling and high-resolution generation tool. It uses tiled diffusion to create ultra-large images by processing them in smaller, overlapping regions to prevent memory crashes on limited hardware. The system manages spatial composition through regional prompting, which routes specific text prompts to designated areas of an image. It maintains visual stability and global coherence during the upscaling process using noise inversion and structural guidance. Additional capabilities include tiled detail upscaling and memory optimization for the variational autoencoder

Pulse is a generative model image upscaler and latent space image processor. It functions as a self-supervised photo upsampling tool that increases image resolution by exploring the latent space of pre-trained generative models to synthesize high-quality details. The system includes a face image alignment tool designed to standardize the scale and orientation of raw facial photos. This preprocessing utility prepares images for higher resolution processing by aligning and downscaling faces to a standard orientation. The project covers AI image super-resolution and generative photo upscaling,

This project is a diffusion model training framework and image synthesis pipeline. It provides the tools necessary to train generative models to learn image data distributions through an iterative denoising process. The framework includes a generative model evaluation tool consisting of automated scripts used to measure the quality and accuracy of produced samples. The system covers model training pipelines and performance evaluation for generative diffusion models.