LMArena Agent Arena Chat Bash Reliability
Agents · 2026-08-14
Signed percentage-point Agent Arena Chat category diagnostic signal for successful bash commands, excluding failures outside the model's control, recomputed on sessions tagged as general agentic chat such as creative writing, learning, everyday research and media generation. Atlas keeps the publisher's raw bash_reliability_no_prereq sign, so values are baseline-relative, not absolute success probabilities, and may be negative relative to the category baseline; reported confidence intervals are not standard errors.
Top models (higher is better)
| Model | Score |
|---|---|
| GPT-5.5 | 15.6 |
| Claude Opus 5.5 | 15.1 |
| Claude Fable 5 | 15.0 |
| GPT-6 Astra | 14.5 |
| Claude Fable 5.1 | 13.9 |
| GPT-6 Sol | 13.4 |
| Claude Opus 5 | 13.4 |
| Claude Sonnet 5.5 | 13.3 |
| Sonnet 5 | 13.2 |
| Gemini 3.8 Flash | 12.4 |
| Muse Wonder Full | 10.4 |
| GPT-6 Luna | 10.1 |
| Muse Wonder | 9.9 |
| GPT-6.1 Sol | 9.5 |
| GPT-5.6 Sol | 9.5 |
| Gemini 3.7 Flash | 8.8 |
| GPT-5.4 | 8.4 |
| Grok 4.7 | 8.2 |
| Grok 4.6 | 8.2 |
| Claude Haiku 5.5 | 7.6 |
| Gemini 3.1 Pro Preview | 7.5 |
| GPT-5.6 Luna | 6.6 |
| MiMo-V2.6-Pro | 5.6 |
| DeepSeek V4.1 Flash 1017 | 5.1 |
| Muse Spark 1.1 | 5.0 |
| Inkling-Small | 4.5 |
| GPT-5.6 Terra | 4.1 |
| Gemini 4 Argon | 3.5 |
| Mistral Large 4 Preview | 3.0 |
| Muse Spark 1.3 | 2.8 |
| MiMo-V2.5-Pro | 2.1 |
| Qwen3.7-Max (source unspecified) | 0.8 |
| DeepSeek V4.1 Flash | -1.3 |
| Qwen3.8-Max | -1.6 |
| DeepSeek V4 Pro 0813 | -2.1 |
| MiMo-V2.6-Flash | -2.3 |
| Kimi K3 | -2.7 |
| Hy4 Preview | -3.7 |
| GLM-5.2 | -3.8 |
| GLM-5.3 | -3.9 |