ZeroBench Subquestions (Inspect, May 2025)
Multimodal · 2025-05-30
UK AISI's Inspect evaluation of the May 30, 2025 ZeroBench subquestion dataset, with 334 subquestions grouped under 100 main-question IDs. The k=1 mean accuracy first averages subquestion correctness within each main question, then averages those 100 means. The published GPT-4o-mini run uses greedy decoding and a 16,000-token cap; its standard error is across main-question means. Keep this historical dataset condition separate from later official grading and question revisions.
Top models (higher is better)
| Model | Score |
|---|---|
| GPT-4o Mini | 24.5 |