SWE-Marathon v1.1
Code
Current 20-task SWE-Marathon cohort with eight trials per task, using closed-network task environments and the GPU post-training IFEval task. Binary pass@1 is averaged across all 160 trials for complete published leaderboard configurations. Kept separate from the archived v1.0 task environments and five-trial aggregates.
Top models (higher is better)
| Model | Score |
|---|---|
| Muse Wonder Full | 53.1 |
| Muse Wonder | 52.5 |
| Claude Sonnet 5.5 | 51.3 |
| Claude Haiku 5.5 | 50.0 |
| Claude Opus 5 | 50.0 |
| Opus 4.8 | 48.8 |
| Kimi K3 | 48.1 |
| Claude Opus 5.5 | 46.3 |
| Muse Spark 1.3 | 46.3 |
| Claude Fable 5.1 | 45.6 |
| Grok 4.7 | 45.6 |
| Gemini 4 Argon | 43.8 |
| GPT-6 Sol | 43.1 |
| GLM-5.3 | 42.5 |
| GPT-5.6 Sol | 42.5 |
| GPT-6.1 Sol | 42.5 |
| GPT-6 Astra | 42.5 |
| GLM-5.3 Flash | 38.8 |
| GPT-5.6 Terra | 32.5 |
| Grok 4.6 | 31.9 |
| Sonnet 5 | 30.0 |
| Grok 4.5 | 29.4 |
| GPT-6 Luna | 28.1 |
| GPT-5.6 Luna | 24.4 |
| Gemini 3.7 Flash | 22.5 |
| GLM-5.2 | 19.4 |