AURORA Multilingual τ³-bench (Persian)
Agents · 2026-09-30
LILT's Persian-locale run of τ³-bench customer-support agents on AURORA, with localized user-simulator prompts, task scenarios and databases, a gemini-3.5-flash user simulator, and reasoning disabled or at its lowest setting. The score is pass^1 over four trials, averaged with equal weight across the airline, retail and telecom base splits and a 20-task banking sample. Kept as a diagnostic: it is one locale of a modified τ³-bench protocol with no published cross-locale aggregate.
Top models (higher is better)
| Model | Score |
|---|---|
| Claude Opus 5.5 | 77.7 |
| GPT-6 Sol | 68.9 |
| Gemini 3.8 Flash | 60.6 |
| GPT-5.6 Sol | 56.4 |
| Mistral Medium 3.5 | 50.7 |