CWE-bench v1 Judge Panel Pass@1
Code · 2026-09-25
Mean rollout success on the 120 CWE-bench v1 tasks under a majority vote of three agentic judges checking correctness, preserved functionality, generalization, integrity and soundness; distinct from the deterministic verifier.
Top models (higher is better)
| Model | Score |
|---|---|
| Claude Opus 5.5 | 67.0 |
| GPT-6 Astra | 66.0 |
| Grok 4.7 | 64.0 |
| Gemini 4 Argon | 62.0 |
| Claude Fable 5.1 | 60.0 |
| Grok 4.6 | 55.0 |
| DeepSeek V4.1 Flash | 53.0 |
| GPT-6 Sol | 52.0 |
| Muse Spark 1.3 | 51.0 |
| Hy4 Preview | 50.0 |
| Inkling | 33.0 |