Awesome GitHub RepositoriesVisual Token Generation

Janus is a multimodal large language model and unified framework that integrates visual understanding and image generation within a single neural network. It functions as both a visual understanding model for analyzing images and a text-to-image generator. The system uses a unified transformer backbone and a multimodal latent space to bridge the gap between text and visual data. This architecture employs decoupled visual encoding and cross-modal tokenization to separate the paths for discriminative understanding and generative tasks, representing images as grids of discrete codes. The projec

VAR is a visual autoregressive model and image generation framework that applies large language model scaling laws to visual data. It functions as an image generator that uses a coarse-to-fine next-scale prediction approach rather than traditional raster-scan tokenization. The system utilizes scale-based tokenization to represent images as a hierarchy of discrete tokens. It generates high-resolution content by iteratively predicting the next resolution level, refining coarse predictions into fine-grained details. The project covers a broad range of capabilities including autoregressive image

Taming Transformers is a generative system for high-resolution image synthesis that combines a vector-quantized GAN image encoder with an autoregressive transformer. It utilizes a discrete latent space to represent images as codebook tokens, enabling the production of high-fidelity visuals through a hybrid architecture. The project provides specialized capabilities for layout-based scene synthesis, allowing for the creation of complex images by placing objects according to defined bounding box coordinates. It also includes tools for image inpainting to fill missing sections of an image by ana

OmniGen2 es un modelo de generación de imágenes unificado y un modelo de lenguaje grande multimodal diseñado para manejar la generación de texto a imagen, tareas de imagen a imagen y edición de imágenes dentro de un único framework. Funciona como un motor visual de modelo de lenguaje causal capaz de generar y editar imágenes basadas en entradas combinadas de texto y visuales. El sistema cuenta con composición visual en contexto y generación impulsada por sujetos, permitiéndole extraer sujetos de imágenes de referencia y colocarlos en nuevas escenas. También admite la edición de imágenes basada en instrucciones, donde objetos o estilos específicos se modifican mediante comandos de lenguaje natural mientras se preserva el resto de la imagen. Las capacidades del modelo se extienden al análisis y razonamiento de contenido visual, permitiendo el reconocimiento de objetos a través de entradas combinadas de texto y visión. Para mejorar la calidad de salida, emplea un proceso de refinamiento visual iterativo con un mecanismo de autocorrección. El rendimiento se gestiona mediante la optimización del uso de VRAM a través de la descarga dinámica de pesos y la aceleración de la velocidad de inferencia utilizando técnicas de caché.

MAGI-1 is an autoregressive video generation model designed to synthesize high-resolution video sequences from text prompts and image references. It functions as a generative system for text-to-video, image-to-video, and video-to-video transformations. The model utilizes an autoregressive architecture that treats spatio-temporal patches as a sequence of discrete tokens to maintain temporal motion. It employs a variational autoencoder to compress the spatial and temporal dimensions of video data and uses distillation-based step scaling to allow for inference budget control. The system integra