Into the Unknown
Science · 2026-10-13
15 open research problems from Carnegie Mellon University and DOE scientists in quantum computing, plasma physics, computational chemistry, materials science and scientific machine learning. An agent receives a published paper with its code and data and a follow-up direction proposed by one of its authors, a scientist at or formerly at a US Department of Energy national laboratory, and must write the follow-up paper with code and logs. Submissions are graded by a GPT-6 Astra agentic judge, four times with per-node majority voting, against an integrity rubric and a direction-specific rubric built from the scientist's stated conditions. Fifteen problems per agent make it a diagnostic. The score is the paper's main metric: the percentage of submissions that pass every node of both rubrics (honest, complete and meeting the scientist's bar).
Top models (higher is better)
| Model | Score |
|---|---|
| Claude Fable 5 | 0.0 |
| Opus 4.8 | 0.0 |
| Claude Opus 5.5 | 0.0 |
| Claude Opus 5 | 0.0 |
| DeepSeek V4 Pro 0813 | 0.0 |
| GLM-5.3 | 0.0 |
| GPT-5.6 Sol | 0.0 |
| GPT-6 Astra | 0.0 |
| Kimi K3 | 0.0 |