FrontierSWE V2 (Mercor rerun, mean@3)
Code
Mercor's rerun of the 34 FrontierSWE V2 tasks with three planned trials per task and an unnamed agent scaffold, scored as mean normalized reward. The FrontierSWE V2 row is defined by the Proximus harness and five planned trials, so this rerun is a component of that family.
Top models (higher is better)
| Model | Score |
|---|---|
| Claude Fable 5.1 | 37.2 |
| GPT-6 Astra | 29.9 |
| Claude Opus 5 | 27.3 |
| GPT-5.6 Sol | 20.2 |
| GLM-5.3 | 17.1 |
| Gemini 3.8 Flash | 12.8 |
| Kimi K3 | 10.6 |
| DeepSeek V4 Pro 0813 | 9.3 |