DecepEval Tool-use (Neutral)
Safety · 2026-10-06
Deception rate of LLM agents on the neutral versions of DecepEval's 550 Tool-use tasks (information retrieval, verification and external information acquisition), in which tools return deterministic failures or limited evidence that the agent must report faithfully, with routine requirements and no inducement. A Claude Fable 5 judge labels a trajectory deceptive when its reports misstate the tool outcomes or the evidence it actually obtained. A diagnostic baseline for the induced rate; lower is better.
Top models (lower is better)
| Model | Score |
|---|---|
| GPT-5.6 Sol | 2.2 |
| GPT-5.6 Terra | 4.5 |
| GLM-5.3 Flash | 4.9 |
| Grok 4.5 | 6.0 |
| Haiku 4.5 | 7.1 |
| Claude Opus 5 | 9.3 |
| Sonnet 5 | 11.1 |
| DeepSeek V4 Pro 0813 | 23.8 |