Benchmarks

← All benchmarks

MirrorCode (ML, +Private, 2L)

Code

Epoch and METR's leaderboard protocol: reimplement 15 medium/large programs, including private targets, in two languages for 30 tasks. Three attempts per task, each with 10 billion tokens and seven days. A solve requires all public and held-out end-to-end tests to pass. Distinct from the paper's 25-program, six-language protocol.

Top models (higher is better)