NeurIPS 2023 Datasets and Benchmarks Track LAMM: Multi-Modal Large Language Models and Applications as AI Agents
Sparkles: Unlocking Chats Across Multiple Images for Multimodal Instruction-Following Models
Data and code for paper "M3Exam: A Multilingual, Multimodal, Multilevel Benchmark for Examining Large Language Models"
(CVPR2024)A benchmark for evaluating Multimodal LLMs using multiple-choice questions.
[ECCV 2024] M3DBench introduces a comprehensive 3D instruction-following dataset with support for interleaved multi-modal prompts.
The main features of openm3d/m3dbench are: Evaluation Benchmarks, Multimodal Benchmarks, Pre-training Datasets.
Open-source alternatives to openm3d/m3dbench include: openlamm/lamm — [NeurIPS 2023 Datasets and Benchmarks Track] LAMM: Multi-Modal Large Language Models and Applications as AI Agents. hypjudy/sparkles — Sparkles: Unlocking Chats Across Multiple Images for Multimodal Instruction-Following Models. fuxiaoliu/lrv-instruction — [ICLR'24] Mitigating Hallucination in Large Multi-Modal Models via Robust Instruction Tuning. ailab-cvc/seed-bench — (CVPR2024)A benchmark for evaluating Multimodal LLMs using multiple-choice questions. freedomintelligence/mllm-bench — MLLM-Bench: Evaluating Multimodal LLMs with Per-sample Criteria. bradyfu/video-mme — ✨✨[CVPR 2025] Video-MME: The First-Ever Comprehensive Evaluation Benchmark of Multi-modal LLMs in Video Analysis.