AURORA Multilingual Terminal-Bench
Agents · 2026-09-30
LILT's multilingual terminal-agent benchmark on its AURORA leaderboard: 324 command-line tasks whose instructions were written by native speakers in ten languages (Arabic, Chinese, Czech, German, Hindi, Japanese, Korean, Serbian, Spanish and Turkish), run in Harbor with the Terminus-2 agent and per-task time limits of at least 600 seconds. The score is the trial-level pass rate pooled over all of a model's attempts (nominally five trials per task). It expands the 300-task Terminal-Bench-LILT paper set, whose scores are not comparable.
Top models (higher is better)
| Model | Score |
|---|---|
| Claude Opus 5.5 | 76.5 |
| Claude Opus 5 | 73.7 |
| Gemini 3.8 Flash | 67.7 |
| GPT-6 Sol | 65.3 |
| GPT-5.6 Sol | 64.5 |
| Muse Spark 1.3 | 62.5 |
| GPT-5.5 | 60.3 |
| Gemini 3.5 Flash | 58.4 |
| Opus 4.8 | 49.1 |
| Mistral Medium 3.5 | 20.0 |