← All benchmarks

Into the Unknown Direction Node Pass Rate

Science · 2026-10-13

Node-level direction-specific grading on the 15 Into the Unknown research problems. An agent receives a published paper with its code and data and a follow-up direction proposed by one of its authors, a scientist at or formerly at a US Department of Energy national laboratory, and must write the follow-up paper with code and logs. Submissions are graded by a GPT-6 Astra agentic judge, four times with per-node majority voting, against an integrity rubric and a direction-specific rubric built from the scientist's stated conditions. Fifteen problems per agent make it a diagnostic. The score is the percentage of direction-specific rubric nodes, covering required deliverables and the scientist's success conditions, that received their highest rating across the agent's submissions.

Top models (higher is better)