Are We on the Right Way for Evaluating Large Vision-Language Models?

Abstract

Large vision-language models (LVLMs) have recently achieved rapid progress, sparking numerous studies to evaluate their multi-modal capabilities. However, we dig into current evaluation works and identify two primary issues: 1) Visual content is unnecessary for many samples. The answers can be directly inferred from the questions and options, or the world knowledge embedded in LLMs. This phenomenon is prevalent across current benchmarks. For instance, GeminiPro achieves 42.9% on the MMMU benchmark without any visual input, and outperforms the random choice baseline across six benchmarks over 24% on average. 2) Unintentional data leakage exists in LLM and LVLM training. LLM and LVLM could still answer some visual-necessary questions without visual content, indicating the memorizing of these samples within large-scale training data. For example, Sphinx-X-MoE gets 43.6% on MMMU without accessing images, surpassing its LLM backbone with 17.9%. Both problems lead to misjudgments of actual multi-modal gains and potentially misguide the study of LVLM. To this end, we present MMStar, an elite vision-indispensable multi-modal benchmark comprising 1,500 samples meticulously selected by humans. MMStar benchmarks 6 core capabilities and 18 detailed axes, aiming to evaluate LVLMs' multi-modal capacities with carefully balanced and purified samples. These samples are first roughly selected from current benchmarks with an automated pipeline, human review is then involved to ensure each curated sample exhibits visual dependency, minimal data leakage, and requires advanced multi-modal capabilities. Moreover, two metrics are developed to measure data leakage and actual performance gain in multi-modal training. We evaluate 16 leading LVLMs on MMStar to assess their multi-modal capabilities, and on 7 benchmarks with the proposed metrics to investigate their data leakage and actual multi-modal gain.

Author abstract · arXiv abstract · v2

Publication

Neural Information Processing Systems (NeurIPS), 2024

Paper and resources

• Paper · arXiv v2 (PDF)
• arXiv:2403.20330
• Google Scholar
• Code · MMStar-Benchmark/MMStar
• Project website

Research topics

MMStar · Multimodal evaluation · Visual necessity · Data leakage · Language shortcuts · Vision-language benchmarks · Multimodal gain · Benchmark contamination

Research problem and approach

Many multimodal benchmark questions can be answered without images, and training-set contamination can further inflate scores. MMStar combines automated screening with human review to evaluate visual dependence and actual gains from multimodal training. arXiv abstract · v2

Main contributions

• Curates 1,500 questions covering six core capabilities and 18 detailed axes. arXiv abstract · v2
• Introduces metrics for data leakage and multimodal performance gain, evaluated across seven benchmarks. arXiv abstract · v2

Method comparison

arXiv abstract · v2

Selected results

• On six existing multimodal benchmarks, GPT-4V scores 41.2 on average when images are removed, versus 66.0 with images and 35.3 for its GPT-4-Turbo language backbone. The control measures how much benchmark performance remains without visual input. Table 3 · image-removal control · arXiv v2
• On MMStar with two-shot text-only inference, Gemini-Pro scores 20.6% and GPT-4-Turbo 12.2%, compared with the reported random-choice baseline of 24.6%. This supports the benchmark’s requirement for visual evidence under the tested prompting protocol. Table 4 · MMStar text-only control · arXiv v2

Cite this paper

Lin Chen, Jinsong Li, Xiaoyi Dong, Pan Zhang, Yuhang Zang, Zehui Chen, Haodong Duan, Jiaqi Wang, Yu Qiao, Dahua Lin, Feng Zhao. Are We on the Right Way for Evaluating Large Vision-Language Models?. Advances in Neural Information Processing Systems, 37, 2024, pp. 27056–27087.

@inproceedings{arxiv240320330,
  title     = {{Are We on the Right Way for Evaluating Large Vision-Language Models?}},
  author    = {Lin Chen and Jinsong Li and Xiaoyi Dong and Pan Zhang and Yuhang Zang and Zehui Chen and Haodong Duan and Jiaqi Wang and Yu Qiao and Dahua Lin and Feng Zhao},
  booktitle = {Advances in Neural Information Processing Systems},
  year      = {2024},
  volume    = {37},
  publisher = {Curran Associates, Inc.},
  pages     = {27056--27087},
  doi       = {10.52202/079017-0850},
  url       = {https://papers.nips.cc/paper_files/paper/2024/hash/2f8ee6a3d766b426d2618e555b5aeb39-Abstract-Conference.html}
}