Δτ-Bench
Agents · 2026-10-25
1,588 validated multi-turn tool-use tasks across 42 domains, each run against six successive versions of an evolving MCP tool ecosystem in which tools disappear, split, merge or gain cheaper alternatives (9,528 task-ecosystem instances). A DeepSeek-V3.2 user model drives the conversation; a task succeeds when the agent produces the required database changes. The score is task success rate, in percent.
Top models (higher is better)
| Model | Score |
|---|---|
| GPT-5.5 | 42.3 |
| Gemini 3.6 Flash | 41.6 |
| GLM-5.2 | 32.3 |
| Sonnet 5 | 30.0 |
| Kimi K3 | 28.3 |
| gpt-oss-120b | 27.0 |