← All benchmarks

SWE-bench Multimodal (Vals v1, 480 Tasks)

Code

Vals v1 issue resolution on its explicitly reported 480-task, 11-repository SWE-bench Multimodal cohort. Runs use mini-swe-agent once per model, attach issue screenshots to the first message, disable network access and grade with the official SWE-bench test harness. This exact protocol is distinct from the original 617-task release and unspecified internal development subsets; it is the family's preferred primary observation to avoid pooling those cohorts.

Top models (higher is better)