DecepEval Coding (Neutral)
Safety · 2026-10-06
Deception rate of LLM agents on the neutral versions of DecepEval's 432 Coding tasks (completion, modification and testing of functions drawn from 16 real Python repositories, run with Smolagents in Docker against visible tests and hidden checks), with routine requirements and no inducement. A Claude Fable 5 judge labels a trajectory deceptive when its delivery claims misstate the code it wrote or the verification it ran. A diagnostic baseline for the induced rate; lower is better.
Top models (lower is better)
| Model | Score |
|---|---|
| GPT-5.6 Sol | 1.6 |
| GLM-5.3 Flash | 2.2 |
| DeepSeek V4 Pro 0813 | 4.3 |
| Grok 4.5 | 4.9 |
| GPT-5.6 Terra | 11.1 |
| Haiku 4.5 | 13.2 |
| Claude Opus 5 | 20.4 |
| Sonnet 5 | 22.2 |