Decision-Bench v1.1
Professional Work · 2026-10-01
TextQL's 210 enterprise data-agent tasks over a simulated food-delivery platform's 235-table ERP warehouse, run with the publisher's Inspect AI reference agent. Agents file decisions such as bans, journal entries and forecasts, each graded 0-100 by its consequences in the simulator; the score is the mean task score. Version 1.1 re-graded every run under a revised forecast rule, so v1.0 paper scores are not this cohort. The publisher's v1.2 (October 19) keeps the same tasks, agent and rule and only corrects three grading bugs, re-grading the affected runs; those corrections are applied in place.
Top models (higher is better)
| Model | Score |
|---|---|
| Claude Opus 5.5 | 59.1 |
| Gemini 4 Argon | 54.5 |
| GPT-6 Astra | 51.5 |
| Claude Sonnet 5.5 | 51.2 |
| GPT-6.1 Sol | 49.1 |
| GPT-6 Sol | 36.5 |
| Muse Wonder | 34.6 |
| Kimi K3 | 28.0 |
| Gemini 3.8 Flash | 26.9 |
| Claude Haiku 5.5 | 25.3 |
| DeepSeek V4.1 Flash | 25.3 |
| GLM-5.3 Flash | 21.3 |
| Muse Spark 1.3 | 19.8 |
| Sonnet 5 | 16.9 |
| GPT-6 Luna | 14.5 |
| Haiku 4.5 | 5.0 |