SWE-Marathon v1.1 (Kimi K3 H20 calibration)
Code · 2026-07-16
Kimi K3 release evaluation on an H20-calibrated branch of SWE-Marathon v1.1's 20 tasks. GPU-task Docker images, performance gates and reference oracles were recalibrated for H20, while correctness and anti-cheat validators were unchanged. The source does not disclose the repeat count; preserve its reported pass@1 separately from ordinary v1.0 and v1.1 results.
Top models (higher is better)
| Model | Score |
|---|---|
| Kimi K3 | 42.0 |
| Opus 4.8 | 40.0 |
| GPT-5.6 Sol | 39.0 |