GeoBench ACW Country Accuracy
Multimodal · 2025-03-22
GeoBench ACW Country Accuracy is the percentage of the 100 ACW (A Community World) locations for which the model names the correct country, from the same runs as the GeoBench headline score. Refused rounds count as misses.
Top models (higher is better)
| Model | Score |
|---|---|
| Claude Fable 5 | 89.0 |
| Gemini 3 Flash Preview | 88.0 |
| Gemini 2.5 Pro Preview 05-06 | 86.0 |
| Gemini 3.8 Flash | 86.0 |
| Muse Spark 1.3 | 86.0 |
| Gemini 3.5 Flash | 85.0 |
| Claude Fable 5.1 | 84.0 |
| Gemini 3.1 Pro Preview | 84.0 |
| Gemini 3 Pro Preview | 84.0 |
| Gemini 3.1 Flash-Lite Preview | 83.0 |
| Opus 4.8 | 81.0 |
| GPT-5 | 81.0 |
| Gemini 2.5 Pro Experimental 03-25 | 78.0 |
| Gemini 2.0 Flash | 77.0 |
| Gemini 1.5 Flash (source unspecified) | 76.0 |
| Gemini 2.5 Flash | 76.0 |
| Opus 4.5 | 75.0 |
| Sonnet 5 | 74.0 |
| o3 | 74.0 |
| Sonnet 4.6 | 73.0 |
| Gemini 2.5 Flash Preview 04-17 | 73.0 |
| GPT-5.6 Sol | 73.0 |
| GPT-4.1 | 72.0 |
| GLM-4.6V | 71.0 |
| GPT-4o (source unspecified) | 71.0 |
| GPT-4o Mini | 64.0 |
| O4 Mini | 64.0 |
| Claude 3.5 Sonnet (Oct 2024) | 62.0 |
| Qwen2.5-VL-72B-Instruct | 62.0 |
| Grok 4.7 | 61.0 |
| Sonnet 4.5 | 60.0 |
| Qwen3-VL-235B-A22B-Instruct | 59.0 |
| Grok 4.3 | 56.0 |
| Mistral Medium 3.5 | 56.0 |
| Gemma 3 27B IT | 52.0 |
| Llama 4 Maverick Instruct | 52.0 |
| Opus 4 | 49.0 |
| Grok 4 | 45.0 |
| Qwen3.8 27B | 38.0 |
| Sonnet 4 | 37.0 |