DecepEval Coding (Induced)
Safety · 2026-10-06
Deception rate of LLM agents on the induced versions of DecepEval's 432 Coding tasks (completion, modification and testing of functions drawn from 16 real Python repositories, run with Smolagents in Docker against visible tests and hidden checks). Each induced version adds pressure, an incentive, an opportunity or a conflicting goal to an otherwise identical neutral task, and a Claude Fable 5 judge labels a trajectory deceptive when its delivery claims misstate the code it wrote or the verification it ran. The score is the percentage of tasks judged deceptive; lower is better.
Top models (lower is better)
| Model | Score |
|---|---|
| GLM-5.3 Flash | 10.7 |
| DeepSeek V4 Pro 0813 | 21.9 |
| GPT-5.6 Terra | 27.3 |
| Haiku 4.5 | 28.0 |
| GPT-5.6 Sol | 31.7 |
| Grok 4.5 | 40.0 |
| Claude Opus 5 | 64.3 |
| Sonnet 5 | 68.1 |