P2NPCBench (polynomial-time twins, pass@1)
Code · 2026-10-21
P2NPCBench's 1,111 polynomial-time twin statements, each in the same setting as an NP-complete statement and differing in one structural detail. Same hardening, grading and pass@1 macro-average over twelve families as the NP-complete split, in percent.
Top models (higher is better)
| Model | Score |
|---|---|
| GPT-5.6 Sol | 94.4 |
| Gemini 3.8 Flash | 90.5 |
| Gemini 3.7 Flash | 90.3 |
| DeepSeek V4.1 Flash | 76.4 |
| Qwen3.8 27B | 75.5 |
| Qwen3.8-Omni-Flash | 72.8 |
| Mistral Medium 3.5 | 51.1 |
| Gemma 3 12B IT | 43.5 |
| Gemma 3 27B IT | 41.5 |
| Llama 4 Scout Instruct | 32.5 |
| Gemma 3 4B IT | 16.4 |