ATLAS (Exa, 200-task subset)
Search · 2026-10-08
Exa's ATLAS asks an agent to return a table of every entity that meets a set of conditions, with 2 to 10 attributes each, so it must search wide to find the entities and deep to fill every cell. This row is the fixed seed-42 subset of 200 of ATLAS's 547 tasks, run by Exa with every model in its Scout agent with Exa search and page fetch for up to 25 tool-call rounds and an hour per task. A returned row counts only if the entity and every graded cell are correct; the score is row F1 averaged over tasks.
Top models (higher is better)
| Model | Score |
|---|---|
| GPT-6.1 Sol | 69.5 |
| Gemini 4 Argon | 53.8 |
| Claude Sonnet 5.5 | 46.8 |
| Muse Spark 1.3 | 42.7 |
| Kimi K3 | 42.1 |
| GLM-5.3 | 39.8 |
| Grok 4.7 | 39.5 |
| DeepSeek V4.1 Flash 1017 | 31.9 |