Benchmarks

← All benchmarks

MM-Vet (OpenVLM)

Multimodal

MM-Vet evaluates integrated multimodal capabilities with open-ended answers graded by an LLM judge on a partial-credit scale, so the score is a mean of per-item grades rather than a pass rate. Scored by the OpenVLM Leaderboard, which runs every model through OpenCompass's VLMEvalKit under one fixed harness. That is a different measurement protocol from the lab-reported and Artificial Analysis runs of the same underlying benchmarks, so it forms its own factor group. No evaluation_item_count is declared: the leaderboard publishes scores to one decimal place, which is coarser than the item lattice, so the denominator cannot be confirmed from the numbers and a binomial noise model is not claimed.

Top models (higher is better)