CRED v0.92 - Paper–code
Science · 2026-07-20
The CRED v0.92 paper–code inconsistency chapter measures detection recall on errors that become visible only by comparing claims in an empirical policy paper with its analysis code.
Top models (higher is better)
| Model | Score |
|---|---|
| GPT-5.6 Sol | 98.9 |
| GPT-5.6 Terra | 95.6 |
| Kimi K3 | 95.6 |
| GPT-5.5 | 94.4 |
| GPT-5.6 Luna | 93.3 |
| Claude Fable 5 | 92.2 |
| GLM-5.2 | 80.0 |
| Kimi K2.6 | 75.6 |
| DeepSeek-V4-Pro | 71.1 |
| Opus 4.8 | 71.1 |
| DeepSeek-V4-Flash | 67.8 |
| Sonnet 4.6 | 64.4 |
| Nemotron 3 Ultra 550B A55B | 56.7 |
| Opus 4 | 55.6 |
| Opus 4.1 | 55.6 |
| Sonnet 4.5 | 55.6 |
| Sonnet 5 | 55.6 |
| MiniMax M3 | 54.4 |
| Opus 4.5 | 50.0 |
| GLM-5 | 46.7 |
| Kimi K2 Thinking | 44.4 |
| Kimi K2.5 | 44.4 |
| Kimi K2 Instruct | 41.1 |
| Ring 2.6 1T | 41.1 |
| GLM-4.7 | 36.7 |
| Gemini 3.5 Flash | 36.7 |
| DeepSeek-R1-0528 | 32.2 |
| Qwen3.5 397B A17B | 31.1 |
| Hy3 preview | 26.7 |
| GLM-4.5 | 25.6 |
| GLM 4.6 | 25.6 |
| gpt-oss-120b | 22.2 |
| DeepSeek-V3.2 | 22.2 |
| Nemotron 3 Super 120B A12B | 20.0 |
| Mistral Large 3 675B Instruct 2512 | 18.9 |
| DeepSeek-R1 | 15.6 |
| DeepSeek-V3.1 | 14.4 |
| Gemma 3 27B IT | 10.0 |
| MiniMax M2 | 7.8 |
| DeepSeek-V3-0324 | 5.6 |