2 dépôts
Training processes that map images and text across multiple languages into a shared vector space.
Distinct from Caption-Based Training: Extends caption-based training to include shared latent spaces for multilingual alignment, not just image-text association.
Explore 2 awesome GitHub repositories matching artificial intelligence & ml · Multilingual Image-Text Alignment. Refine with filters or upvote what's useful.
Kolors is a generative model implementation for synthesizing photorealistic images from natural language descriptions and visual references. It utilizes a latent diffusion model framework to produce high-fidelity imagery, operating within a compressed latent space to improve generation efficiency and quality. The system functions as a multilingual image generator, interpreting text prompts in multiple languages to produce semantically accurate visual outputs. It includes a custom model training pipeline that uses low-rank adaptation to teach the model specific subjects or artistic styles from
Provides the ability to interpret text prompts in multiple languages to produce semantically accurate visual outputs.
This project is a research framework and toolkit designed for training large-scale vision transformers and multimodal language models. It provides a comprehensive suite for vision-language pretraining, enabling the development of models that map images and text into shared latent spaces. The framework is distinguished by its capabilities in high-fidelity image generation and multimodal research, utilizing normalizing flows and variational autoencoders to produce images from text prompts or class labels. It supports the development of both generative and contrastive models, allowing for a wide
Maps images and text into a shared space using captioning-based pretraining and self-supervised losses.