AURORA Multilingual τ³-bench (Korean)
Agents · 2026-09-30
LILT's Korean-locale run of τ³-bench customer-support agents on AURORA, with localized user-simulator prompts, task scenarios and databases, a gemini-3.5-flash user simulator, and reasoning disabled or at its lowest setting. The score is pass^1 over four trials, averaged with equal weight across the airline, retail and telecom base splits and a 20-task banking sample. Kept as a diagnostic: it is one locale of a modified τ³-bench protocol with no published cross-locale aggregate.
Top models (higher is better)
| Model | Score |
|---|---|
| Claude Opus 5.5 | 78.4 |
| GPT-6 Sol | 71.0 |
| GPT-5.6 Sol | 63.1 |
| Gemini 3.8 Flash | 60.5 |
| Mistral Medium 3.5 | 51.6 |