GPQA (Open LLM Leaderboard v2)
General QA · 2024-06-26
Graduate-level expert-written science questions. The Open LLM Leaderboard v2 figure pools the main, extended, and diamond splits into one 1,192-question set (confirmed by the reported score lattice), which is a broader and easier set than the 198-question diamond split reported elsewhere. Scored by HuggingFace's Open LLM Leaderboard v2 under lm-evaluation-harness with a fixed prompt template and no chain-of-thought, which is a different measurement protocol from lab-reported and Artificial Analysis runs of the same underlying benchmark, so it forms its own factor group rather than pooling with them. Raw accuracy is imported, not the leaderboard's baseline-normalized column.