LMArena Agent Arena Code Bash Reliability
Agents · 2026-08-14
Signed percentage-point Agent Arena Code category diagnostic signal for successful bash commands, excluding failures outside the model's control, recomputed on sessions tagged as writing or debugging code, automating workflows or analyzing data. Atlas keeps the publisher's raw bash_reliability_no_prereq sign, so values are baseline-relative, not absolute success probabilities, and may be negative relative to the category baseline; reported confidence intervals are not standard errors.
Top models (higher is better)
| Model | Score |
|---|---|
| Claude Opus 5.5 | 15.2 |
| Sonnet 5 | 14.7 |
| GPT-5.5 | 12.0 |
| Claude Sonnet 5.5 | 11.9 |
| GPT-5.6 Sol | 11.5 |
| Claude Fable 5 | 10.7 |
| GPT-6 Luna | 10.4 |
| GPT-6 Sol | 10.1 |
| Claude Opus 5 | 10.1 |
| Gemini 3.8 Flash | 10.1 |
| Grok 4.6 | 10.0 |
| Claude Fable 5.1 | 9.7 |
| Grok 4.7 | 9.2 |
| GPT-5.4 | 9.1 |
| GPT-5.6 Luna | 7.9 |
| Claude Haiku 5.5 | 6.7 |
| Muse Wonder Full | 6.6 |
| Muse Wonder | 6.4 |
| GPT-6 Astra | 6.4 |
| Gemini 3.7 Flash | 6.0 |
| DeepSeek V4 Pro 0813 | 5.0 |
| DeepSeek V4.1 Flash 1017 | 4.7 |
| Qwen3.7-Max (source unspecified) | 4.7 |
| Qwen3.7-Plus (source unspecified) | 4.5 |
| MiMo-V2.6-Pro | 3.1 |
| Muse Spark 1.3 | 2.7 |
| Gemini 3.1 Pro Preview | 2.0 |
| Inkling | 2.0 |
| MiniMax M3 | 1.8 |
| Inkling-Small | 1.7 |
| GPT-5.6 Terra | 1.2 |
| GPT-6.1 Sol | 1.0 |
| Qwen3.8-Max | 0.8 |
| MiMo-V2.5-Pro | 0.2 |
| DeepSeek V4.1 Flash | -0.3 |
| Mistral Large 4 Preview | -0.4 |
| Hy4 Preview | -0.9 |
| MiMo-V2.6-Flash | -2.3 |
| Kimi K3 | -3.1 |
| Muse Spark 1.1 | -3.3 |