MATH
Math · 2021-03-05
MATH contains 12,500 competition mathematics problems across all splits. Its canonical scored test split contains 5,000 problems spanning algebra, geometry, number theory, and more, each accompanied by a full step-by-step solution. Models are scored on the percentage of test problems for which they produce the correct final answer. Higher is better.
Top models (higher is better)
| Model | Score |
|---|---|
| Gemini 2.0 Flash | 90.9 |
| Gemini 2.0 Flash-Lite | 86.8 |
| Claude 3.5 Sonnet (Oct 2024) | 78.3 |
| GPT-4 Turbo | 73.4 |
| Claude 3.5 Sonnet (June 2024) | 71.1 |
| Haiku 3.5 | 69.2 |
| Opus 3 | 60.1 |
| Gemini 1.0 Ultra | 53.2 |
| GPT-4 | 52.9 |
| Gemini 1.5 Flash-8B | 35.9 |
| PaLM 2-L | 34.4 |
| Gemini 1.0 Pro | 32.6 |
| Grok 1 | 23.9 |
| Gemini Nano 2 | 22.8 |
| Gemini Nano 1 | 13.5 |