BrokenArXiv 06/2026
Math
This MathArena track evaluates whether models recognize and refuse to prove 54 plausible but false mathematical statements extracted from recent arXiv papers. Scores report the percentage of problems answered correctly.
Top models (higher is better)
| Model | Score |
|---|---|
| Claude Opus 5 | 90.7 |
| GPT-5.5 | 69.4 |
| GPT-5.6 Sol | 67.3 |
| Kimi K3 | 51.9 |
| Muse Spark 1.1 | 50.6 |
| Claude Fable 5 | 47.8 |
| Opus 4.8 | 41.7 |
| Grok 4.5 | 29.3 |
| Gemini 3.1 Pro Preview | 17.6 |
| DeepSeek-V4-Flash | 16.7 |
| Step 3.7 Flash | 15.7 |
| GLM-5.2 | 14.2 |
| Gemini 3.5 Flash | 12.3 |
| Gemini 3.6 Flash | 12.0 |
| Qwen3.6 35B A3B | 3.4 |
| Qwen3.5-2B | 2.8 |