SecPRBench (recall)
Code · 2026-10-24
265 real vulnerability-introducing code changes from popular C, C++, Rust and JavaScript projects (with their later fixed versions as companions). An agent reviews each bug-introducing change, with the reference vulnerability and later fix withheld, and files findings; a Claude Opus 5.5 judge decides whether a finding identifies the introduced vulnerability. One run per task, web tools off, 60-minute timeout. The score is recall, the share of the 265 targets found, in percent.
Top models (higher is better)
| Model | Score |
|---|---|
| Grok 4.6 | 78.1 |
| GPT-6 Sol | 76.2 |
| GLM-5.3 | 74.7 |
| Claude Opus 5.5 | 74.0 |
| Sonnet 5 | 59.2 |
| GPT-5.6 Sol | 55.8 |
| GPT-6 Luna | 55.5 |
| MiniMax M3 | 32.5 |