CWE-bench v1: Python
Code · 2026-09-25
Programmatic pass@1 across the 14 Python tasks in CWE-bench v1, averaging four rollouts per task.
Top models (higher is better)
| Model | Score |
|---|---|
| GPT-6 Astra | 71.0 |
| Grok 4.7 | 68.0 |
| Gemini 4 Argon | 57.0 |
| Grok 4.6 | 57.0 |
| GPT-6 Sol | 55.0 |
| Claude Fable 5.1 | 52.0 |
| Hy4 Preview | 52.0 |
| Muse Spark 1.3 | 52.0 |
| Claude Opus 5.5 | 50.0 |
| DeepSeek V4.1 Flash | 46.0 |
| Inkling | 36.0 |