Projects sharing features with AMBER

This project is a technical curriculum and development guide focused on large language model prompt engineering, fine-tuning, and the creation of retrieval augmented generation applications. It serves as a comprehensive resource for developers to master crafting precise instructions and textual patterns to improve the quality and predictability of model outputs. The material covers the end-to-end workflow of adapting open-source models to specific datasets and integrating language models with vector databases to generate responses based on private information. It also provides a systematic ap

This project serves as an educational resource and guide for prompt engineering, providing a structured methodology for interacting with large language models. It focuses on teaching core strategies to improve the reliability, accuracy, and consistency of model outputs across a variety of natural language processing tasks. The framework emphasizes the use of standardized templates and logical decomposition to manage complex instructions. By implementing techniques such as few-shot context injection, iterative refinement, and delimiter-based segmentation, the project demonstrates how to guide

Data for evaluating GPT-4V

Train-val-test splits of GPT2Shape dataset can be found in folder gpt2shape: train val test

(CVPR2024)A benchmark for evaluating Multimodal LLMs using multiple-choice questions.

In Stage 1, the Keyword Extraction Prompt instructs ChatGPT to generate verbs, nouns, and adjectives (highlighted in brown) from the original caption. In Stage 2, the Caption Generation Prompt guides ChatGPT to generate a rewritten caption. By iteratively applying this prompt, multiple rewritten…

This repo provides the source code & data of our paper: Evaluating Object Hallucination in Large Vision-Language Models (EMNLP 2023).

This is the official repository of our paper ScanQA: 3D Question Answering for Spatial Scene Understanding (CVPR 2022) by Daichi Azuma, Taiki Miyanishi, Shuhei Kurita, and Motoki Kawanabe. We propose a new 3D spatial understanding task for 3D question answering (3D-QA). In the 3D-QA task, models…

This repository contains the code for the paper "Tricking LLMs into Disobedience: Formalizing, Analyzing, and Detecting Jailbreaks" by Abhinav Rao, Sachin Vashistha, Atharva Naik, Somak Aditya, and Monojit Choudhury, accepted at LREC-CoLING 2024

.svg" alt="Documentation Status">

✨✨CVPR 2025 Video-MME: The First-Ever Comprehensive Evaluation Benchmark of Multi-modal LLMs in Video Analysis

Authors: Liwei Jiang, Kavel Rao ⭐, Seungju Han ⭐, Allyson Ettinger, Faeze Brahman, Sachin Kumar, Niloofar Mireshghallah, Ximing Lu, Maarten Sap, Yejin Choi, Nouha Dziri ⭐ Co-second authors

ECCV 2024 BenchLMM: Benchmarking Cross-style Visual Capability of Large Multimodal Models

✨✨Woodpecker: Hallucination Correction for Multimodal Large Language Models

📰 Latest News 📰 - 🗡️ What is HarmBench 🛡️ - 🌐 Overview 🌐 - ☕ Quick Start ☕ - ⚙️ Installation - 🛠️ Running the Evaluation Pipeline - ➕ Using your own models in HarmBench - ➕ Using your own red teaming methods in HarmBench - 🤗 Classifiers - ⚓ Documentation ⚓ - 🌱 HarmBench's Roadmap 🌱 -…

ICLR 2025 ChartMimic: Evaluating LMM’s Cross-Modal Reasoning Capability via Chart-to-Code Generation

NAACL 2025 Source code for MMEvalPro, a more trustworthy and efficient benchmark for evaluating LMMs

The jailbreak-evaluation is an easy-to-use Python package for language model jailbreak evaluation. The jailbreak-evaluation is designed for comprehensive and accurate evaluation of language model jailbreak attempts. Currently, jailbreak-evaluation support evaluating a language model jailbreak…

🍎 Project Page 📖 arXiv Paper 📊 Dataset🏆 Leaderboard

Data and code for paper "M3Exam: A Multilingual, Multimodal, Multilevel Benchmark for Examining Large Language Models"

CVPR 2024 Highlight Mitigating Object Hallucinations in Large Vision-Language Models through Visual Contrastive Decoding

ICML 2024 Official implementation for "HALC: Object Hallucination Reduction via Adaptive Focal-Contrast Decoding"

We introduce the task of dense captioning in 3D scans from commodity RGB-D sensors. As input, we assume a point cloud of a 3D scene; the expected output is the bounding boxes along with the descriptions for the underlying objects. To address the 3D object detection and description problems, we…

English | 简体中文