MMMU (Maj1@32)
Multimodal · 2023-12-19
MMMU validation accuracy using the majority answer over 32 samples. Kept separate from single-sample (pass@1) MMMU.
Top models (higher is better)
| Model | Score |
|---|---|
| Gemini 1.0 Ultra | 62.4 |
Benchmarks
218 sources · 1,060 models × 2,130 benchmarks · 109,661 scoresMultimodal · 2023-12-19
MMMU validation accuracy using the majority answer over 32 samples. Kept separate from single-sample (pass@1) MMMU.
| Model | Score |
|---|---|
| Gemini 1.0 Ultra | 62.4 |