← All benchmarks

LSCBench Function Granularity

Code · 2026-10-10

The 150 LSCBench tasks posed at function granularity, with the functions, class fragments or statements around the fix masked in place and filled from an inline contract of about 250 lines. Each task asks a coding agent, offline in a Harbor container, to implement a functional specification derived from a real CVE in C, C++, Go, Java, JavaScript, PHP or Python; the specification never mentions security. Hidden functional tests and a security suite that replays the original attack run in a separate verifier, and a task counts only when both pass. The score (FUNC∧SEC) is the percentage of these tasks whose code both works and is secure.

Top models (higher is better)