DeepSWE v1.1 (Fireworks, avg@3)
Code
Fireworks' run of DeepSWE v1.1's 113 long-horizon software-engineering tasks across 91 open-source repositories, using mini-swe-agent through Pier in isolated E2B environments with maximum available reasoning and a 9,000-second agent budget. Each model gets three complete passes, and agent timeouts, verifier timeouts, and errored trials all score as failures. The score is the mean of the three run solve rates, kept as a component beside Datacurve's own pooled pass@1.
Top models (higher is better)
| Model | Score |
|---|---|
| Claude Opus 5 | 72.3 |
| GPT-6 Astra | 71.0 |
| Kimi K3 | 70.2 |
| DeepSeek V4.1 Flash 1017 | 69.6 |
| Claude Opus 5.5 | 68.4 |
| GLM-5.3 | 68.1 |
| Gemini 4 Argon | 67.8 |
| GPT-6 Sol | 67.3 |
| Ember-1 | 67.0 |
| GPT-6 Luna | 66.4 |
| DeepSeek V4.1 Flash | 63.4 |
| GPT-5.6 Sol | 62.0 |
| Gemini 3.8 Flash | 61.6 |
| Grok 4.6 | 57.8 |
| Muse Spark Open | 53.4 |
| DeepSeek V4 Pro 0813 | 45.1 |