OTIS Mock AIME 2024-2025 (Epoch legacy model grader)
Math
Epoch's historical OTIS scores using a model grader that saw the reference answer, before the September 2025 switch to answer extraction and exact matching. The publisher reports that the legacy grader could mark blank answers correct; repeated-run means and single runs remain historical diagnostics rather than comparable corrected-protocol observations.
Top models (higher is better)
| Model | Score |
|---|---|
| GPT-5 | 87.2 |
| Grok 4 | 84.0 |
| o3 | 83.9 |
| O4 Mini | 81.7 |
| GPT-5 Mini | 78.3 |
| Grok 3 Mini | 77.8 |
| o3-mini | 76.9 |
| GPT-5 Nano | 74.2 |
| o1 (2024-12-17) | 73.3 |
| Gemini 2.5 Flash Preview 04-17 | 73.1 |
| Sonnet 4 | 71.1 |
| Gemini 2.5 Flash | 70.8 |
| Opus 4.1 | 68.9 |
| DeepSeek-R1-0528 | 66.4 |
| Opus 4 | 64.4 |
| Claude 3.7 Sonnet | 57.8 |
| Gemini 2.0 Flash Thinking Experimental 01-21 | 57.8 |
| Grok 3 (API) | 55.6 |
| DeepSeek-R1 | 53.3 |
| DeepSeek-R1-Distill-Llama-70B | 51.4 |
| o1-mini | 46.9 |
| GPT-4.1 Mini | 44.7 |
| GPT-4.1 | 38.3 |
| DeepSeek-V3-0324 | 37.8 |
| GPT-4.5 | 37.8 |
| Mistral Medium 3 | 32.2 |
| Gemini 2.0 Flash | 31.1 |
| o1 Preview | 31.1 |
| GPT-4.1 Nano | 28.9 |
| Gemini 1.5 Pro 002 | 23.1 |
| Llama 4 Maverick Instruct FP8 | 20.6 |
| Qwen-Plus (2025-01-25) | 17.8 |
| Gemini 1.5 Flash 002 | 16.3 |
| Qwen2.5-Max | 16.1 |
| DeepSeek-V3 | 15.8 |
| Phi-4 | 13.8 |
| Grok 2 1212 | 11.5 |
| Llama 3.1 405B Instruct | 9.7 |
| Claude 3.5 Sonnet (Oct 2024) | 8.5 |
| Qwen2.5 72B Instruct | 8.1 |