How this analysis was created: This summary and feature list were written by an AI model that read the project's README and public documentation pages. Each feature links to the documentation it came from; stars, license and language come straight from the GitHub API. The model does not read the source code, and the analysis is refreshed when the project is re-analysed. Learn more on our About page.
Understanding Complex Videos Relying on Large Language and Vision Models [Project Page] [Paper] The online demo is no longer available, because we released the code for offline demo deployment
The main features of rupertluo/valley are: Multimodal Learning.
Open-source alternatives to rupertluo/valley include: deepmind/deepmind-research — This project is an AI research implementation library and machine learning research repository. It provides a… deepseek-ai/janus — Janus is a multimodal large language model and unified framework that integrates visual understanding and image… fuxiaoliu/lrv-instruction — [ICLR'24] Mitigating Hallucination in Large Multi-Modal Models via Robust Instruction Tuning. hanzhanggit/stackgan — Pytorch implementation. haotian-liu/llava — LLaVA is a multimodal large language model architecture designed to process and interpret both image and text inputs… baai-dcai/visual-instruction-tuning — Scale up visual instruction tuning to millions by GPT-4.
This project is an AI research implementation library and machine learning research repository. It provides a collection of reference code, illustrative implementations, and open-source research datasets used to verify hypotheses and build upon existing models in artificial intelligence. The repository focuses on scientific research reproduction by translating theoretical findings from published papers into executable code. It includes specialized scientific simulation environments designed to test the behavior of autonomous agents and models within controlled settings. The project covers AI
Janus is a multimodal large language model and unified framework that integrates visual understanding and image generation within a single neural network. It functions as both a visual understanding model for analyzing images and a text-to-image generator. The system uses a unified transformer backbone and a multimodal latent space to bridge the gap between text and visual data. This architecture employs decoupled visual encoding and cross-modal tokenization to separate the paths for discriminative understanding and generative tasks, representing images as grids of discrete codes. The projec
ICLR'24 Mitigating Hallucination in Large Multi-Modal Models via Robust Instruction Tuning
Scale up visual instruction tuning to millions by GPT-4.