Projects sharing features with LRV Instruction

Official repo for StableLLAVA

CVPR 2025 Mitigating Object Hallucinations in Large Vision-Language Models with Assembly of Global and Local Attention

Official implementation of "Visually Dehallucinative Instruction Generation: Know What You Don't Know"

NeurIPS 2023 Datasets and Benchmarks Track LAMM: Multi-Modal Large Language Models and Applications as AI Agents

CVPR 2024 Highlight OPERA: Alleviating Hallucination in Multi-Modal Large Language Models via Over-Trust Penalty and Retrospection-Allocation

Less is More: Mitigating Multimodal Hallucination from an EOS Decision Perspective (ACL 2024)

ACL 2024 Logical Closed Loop: Uncovering Object Hallucinations in Large Vision-Language Models. Detect and mitigate object hallucinations in LVLMs by itself through logical closed loops.

ICML 2024 Official implementation for "HALC: Object Hallucination Reduction via Adaptive Focal-Contrast Decoding"

In Stage 1, the Keyword Extraction Prompt instructs ChatGPT to generate verbs, nouns, and adjectives (highlighted in brown) from the original caption. In Stage 2, the Caption Generation Prompt guides ChatGPT to generate a rewritten caption. By iteratively applying this prompt, multiple rewritten…

Sparkles: Unlocking Chats Across Multiple Images for Multimodal Instruction-Following Models

✨✨Woodpecker: Hallucination Correction for Multimodal Large Language Models

NeurIPS 2025 The official repository of "Inst-IT: Boosting Multimodal Instance Understanding via Explicit Visual Prompt Instruction Tuning"

ICML 2025 Official implementation of paper 'Look Twice Before You Answer: Memory-Space Visual Retracing for Hallucination Mitigation in Multimodal Large Language Models'.

Aligning LMMs with Factually Augmented RLHF

ICLR '25 Official Pytorch implementation of "Interpreting and Editing Vision-Language Representations to Mitigate Hallucinations"

VLMEvalKit is a vision-language model evaluation framework and inference engine designed to run standardized benchmarks and measure model accuracy across diverse visual datasets. It serves as a multimodal model benchmark and performance toolkit for calculating metrics and comparing model responses. The toolkit includes a specialized visual reasoning evaluator that uses adversarial samples to distinguish actual image understanding from reliance on language patterns. It also provides capabilities for image generation evaluation, testing a model's ability to create or modify visuals based on tex

ChatLaw is a specialized large language model legal assistant designed to provide automated consulting and question answering within Chinese legal frameworks. It functions as a system for legal knowledge management, processing complex legal texts to deliver accurate statutory answers and advisory services. The system utilizes a mixture-of-experts modeling approach and multi-agent coordination to research information and generate professional consultation reports. To ensure factual reliability and minimize hallucinations, it integrates a legal knowledge graph and a standardized operating proce

This project provides a comprehensive Chinese language corpus designed to support the training and fine-tuning of large language models. It serves as a structured natural language processing resource, offering a collection of text data that includes dialogue, customer service interactions, and creative writing. The dataset is organized into distinct thematic categories, allowing for targeted model development across specific conversational and narrative contexts. By providing information in standardized, schema-agnostic text formats, the collection ensures portability across various machine l

This project serves as an educational resource and guide for prompt engineering, providing a structured methodology for interacting with large language models. It focuses on teaching core strategies to improve the reliability, accuracy, and consistency of model outputs across a variety of natural language processing tasks. The framework emphasizes the use of standardized templates and logical decomposition to manage complex instructions. By implementing techniques such as few-shot context injection, iterative refinement, and delimiter-based segmentation, the project demonstrates how to guide

Diagram of Self-Introspective Decoding.

Paper: PAINT: PAYING ATTENTION TO INFORMED TOKENS TO MITIGATE HALLUCINATION IN LARGE VISION-LANGUAGE MODEL

Fine-tuning Multimodal LLMs to Follow Zero-shot Demonstrative Instructions

LLaVA is a multimodal large language model architecture designed to process and interpret both image and text inputs to generate natural language responses. It functions as a research-oriented platform for visual instruction tuning, providing a framework to align language models with human intent through training on diverse datasets of paired images and text queries. The system distinguishes itself through a specialized vision-language training pipeline that connects visual data to language models using projection layers and instruction-based fine-tuning. It supports distributed inference by

Pytorch implementation