Projects sharing features with RLHF V

This project serves as an educational resource and guide for prompt engineering, providing a structured methodology for interacting with large language models. It focuses on teaching core strategies to improve the reliability, accuracy, and consistency of model outputs across a variety of natural language processing tasks. The framework emphasizes the use of standardized templates and logical decomposition to manage complex instructions. By implementing techniques such as few-shot context injection, iterative refinement, and delimiter-based segmentation, the project demonstrates how to guide

FaithScore: Fine-grained Evaluations of Hallucinations in Large Vision-Language Models

This repository provides supplementary material for our paper Constitutional AI: Harmlessness from AI Feedback.

Fine-Grained RLHF

trlx is a reinforcement learning library and training framework designed to align large language models using human feedback. It serves as a distributed trainer and compute orchestrator for scaling high-parameter models across multiple GPUs and nodes. The project provides tools for reinforcement learning from human feedback and model alignment. It implements reward-model-based optimization and proximal policy optimization to refine model behavior based on goal-oriented rewards or human-labeled datasets. The framework covers distributed training strategies, including model parallelism, parame

CVPR 2024 Highlight Mitigating Object Hallucinations in Large Vision-Language Models through Visual Contrastive Decoding

In Stage 1, the Keyword Extraction Prompt instructs ChatGPT to generate verbs, nouns, and adjectives (highlighted in brown) from the original caption. In Stage 2, the Caption Generation Prompt guides ChatGPT to generate a rewritten caption. By iteratively applying this prompt, multiple rewritten…

Arxiv

✨✨Woodpecker: Hallucination Correction for Multimodal Large Language Models

ACL 2024 Logical Closed Loop: Uncovering Object Hallucinations in Large Vision-Language Models. Detect and mitigate object hallucinations in LVLMs by itself through logical closed loops.

Official PyTorch Implementation for the "What if...?: Thinking Counterfactual Keywords Helps to Mitigate Hallucination in Large Multi-modal Models" paper (EMNLP Findings 2024).

An LLM-free Multi-dimensional Benchmark for Multi-modal Hallucination Evaluation

An automatic MLLM hallucination detection framework

ICLR'24 Mitigating Hallucination in Large Multi-Modal Models via Robust Instruction Tuning

CVPR 2025 Mitigating Object Hallucinations in Large Vision-Language Models with Assembly of Global and Local Attention