GeoBench ACW Country Accuracy
Multimodal · 2025-03-22
GeoBench ACW Country Accuracy measures whether a model names the correct country for each image in the A Common World evaluation set. It is a percentage-based classification metric distinct from GeoBench's distance-derived 0-5000 point score; reported runs may evaluate different-sized subsets of the full set.
Top models (higher is better)
| Model | Score |
|---|---|
| Gemini 3 Flash Preview | 88.0 |
| Gemini 2.5 Pro Preview 05-06 | 86.0 |
| Gemini 3 Pro Preview | 84.0 |
| Gemini 2.5 Pro Experimental 03-25 | 81.0 |
| GPT-5 | 81.0 |
| O1 | 80.0 |
| Gemini 2.0 Flash 001 | 77.0 |
| Gemini 1.5 Flash 002 | 76.0 |
| Gemini 2.5 Flash Preview 05-20 | 76.0 |
| Opus 4.5 | 75.0 |
| o3 | 74.0 |
| Gemini 2.5 Flash Preview 04-17 | 73.0 |
| GPT-4.1 | 72.0 |
| GPT-4o (2024-11-20) | 71.0 |
| Claude 3.7 Sonnet | 68.0 |
| GPT-4o Mini | 64.0 |
| O4 Mini | 64.0 |
| Claude 3.5 Sonnet (Oct 2024) | 62.0 |
| Qwen2.5-VL-72B-Instruct | 62.0 |
| Gemma 3 27B IT | 52.0 |
| Llama 3.2 90B Vision Instruct | 52.0 |
| Llama 4 Maverick Instruct | 52.0 |
| Opus 4 | 49.0 |
| Grok 4 | 45.0 |
| Sonnet 4 | 37.0 |
| Pixtral 12B | 37.0 |
| Haiku 3.5 | 34.0 |