MathVista (mini)
Multimodal · 2023-10-03
MathVista-Mini is the testmini evaluation subset of MathVista, covering mathematical reasoning in diverse visual contexts. Scores report answer accuracy.
Top models (higher is better)
| Model | Score |
|---|---|
| Kimi K1.5 | 74.9 |
| O1 | 73.9 |
| Llama 4 Maverick Instruct | 73.7 |
| Gemini 2.0 Flash | 73.1 |
| QVQ 72B Preview | 71.4 |
| Qwen2 VL 72B Instruct | 70.5 |
| Pixtral Large | 69.4 |
| Grok 2 | 69.0 |
| Claude 3.5 Sonnet (June 2024) | 67.7 |
| Gemini 1.5 Pro (May 2024) | 63.9 |
| GPT-4o | 63.8 |
| Phi-4-multimodal-instruct | 62.4 |
| Gemini 1.5 Flash Preview 0514 | 58.4 |
| Gemini 1.5 Flash | 58.4 |
| GPT-4 Turbo | 58.1 |
| Pixtral 12B | 58.0 |
| Gemini 1.0 Ultra | 53.0 |
| Opus 3 | 50.5 |
| GPT-4 Turbo with Vision | 49.9 |
| Claude 3 Sonnet | 47.9 |
| Gemini 1.0 Pro | 46.6 |
| Claude 3 Haiku | 46.4 |
| Gemini 1.0 Pro Vision | 45.2 |
| Phi-3.5-vision-instruct | 43.9 |
| Gemini Nano 2 | 30.6 |
| Gemini Nano 1 | 27.3 |