FrontierCode 2.0 (Artificial Analysis)
Code · 2026-10-27
Artificial Analysis' implementation of Cognition's FrontierCode 2.0 on all 132 private tasks in real open-source repositories. Stirrup runs three attempts per task with a 500-turn cap and background shell commands, and GPT-6 Luna (medium) judges each attempt against the task's tests and a rubric of blocking and non-blocking checks. Attempts that fail a blocking check or are flagged for looking up the solution score zero; others receive a weighted rubric score, averaged equally over tasks and attempts. This evaluator protocol is kept separate from Cognition's native runs.