Benchmarks

← All benchmarks

MMStar (OpenVLM)

Multimodal

MMStar is a curated multiple-choice benchmark of vision-indispensable samples, filtered so that questions cannot be answered from text alone. Accuracy over the whole set. Scored by the OpenVLM Leaderboard, which runs every model through OpenCompass's VLMEvalKit under one fixed harness. That is a different measurement protocol from the lab-reported and Artificial Analysis runs of the same underlying benchmarks, so it forms its own factor group. No evaluation_item_count is declared: the leaderboard publishes scores to one decimal place, which is coarser than the item lattice, so the denominator cannot be confirmed from the numbers and a binomial noise model is not claimed.

Top models (higher is better)