SimpleQA
General QA · 2024-10-30
Short factual question-answering benchmark with single verifiable answers.
Top models (higher is better)
| Model | Score |
|---|---|
| Gemini 2.5 Pro | 54.0 |
| Gemini 2.5 Pro Experimental 03-25 | 52.9 |
| Gemini 2.5 Pro Preview 05-06 | 50.8 |
| o3 | 48.6 |
| Grok 3 | 43.6 |
| DeepSeek-R1 | 30.1 |
| Gemini 2.0 Flash | 29.9 |
| Gemini 2.5 Flash Preview 04-17 | 29.7 |
| DeepSeek-R1-0528 | 27.8 |
| Gemini 2.5 Flash | 26.9 |
| Gemini 2.5 Flash Preview (09-2025) | 25.3 |
| Gemini 1.5 Pro | 24.9 |
| Gemini 2.0 Flash-Lite | 21.7 |
| O4 Mini | 19.3 |
| Gemini 2.5 Flash-Lite | 13.0 |
| Gemini 1.5 Flash | 8.6 |