Android Bench 2.0
Code · 2026-09-17
Long-horizon Android development on 30 private tasks, evaluated in five independent trials per model. Pass rate averages fully successful tasks across trials; deterministic checks and visual judging assess the final app. Harnesses vary by model. Methodology and live task tables disagree on creation versus feature category counts, so only aggregate metrics are imported.
Top models (higher is better)
| Model | Score |
|---|---|
| Claude Opus 5.5 | 32.7 |
| GPT-6 Astra | 28.0 |
| Claude Fable 5.1 | 22.7 |
| GPT-6 Sol | 18.0 |
| Kimi K3 | 12.0 |
| GPT-6 Luna | 10.7 |
| Gemini 3.8 Flash | 8.0 |
| Gemini 3.7 Flash | 7.3 |
| Sonnet 5 | 6.7 |