Benchmarks

← All benchmarks

MMMU Validation (OpenVLM)

Multimodal

The validation split of MMMU, college-level multimodal questions across six disciplines mixing multiple-choice and open-ended answers. Because the answer formats differ across items, no single guessing floor is well defined. Scored by the OpenVLM Leaderboard, which runs every model through OpenCompass's VLMEvalKit under one fixed harness. That is a different measurement protocol from the lab-reported and Artificial Analysis runs of the same underlying benchmarks, so it forms its own factor group. No evaluation_item_count is declared: the leaderboard publishes scores to one decimal place, which is coarser than the item lattice, so the denominator cannot be confirmed from the numbers and a binomial noise model is not claimed.

Top models (higher is better)