Benchmarks

← All benchmarks

Ramp SWE-Bench

Code · 2026-06-11

Ramp SWE-Bench is a private, production-grounded coding benchmark built from real Ramp backend engineering tasks. It evaluates background coding agents using mini-swe-agent on 80 human-accepted tasks, with pass rate reported as the percentage of tasks resolved. The dashboard's task manifest lists all 80 tasks, of which two (tasks 35 and 75) are currently unpublished, so published runs report over a 78-task denominator; evaluation_item_count records the benchmark's canonical 80-task size.

Top models (higher is better)