← All benchmarks

Ramp SWE-Bench (78 published tasks)

Code

Ramp SWE-Bench's published 78-task cohort, comprising the canonical 80-task manifest except task IDs 35 and 75. The primary results CSV contains the same 78 tasks for every reported model, and pass rate is successful tasks divided by 78. These mini-swe-agent runs have an exact cohort identity separate from the complete 80-task benchmark.

Top models (higher is better)