o1 (source unspecified)

Name: o1 (source unspecified)
Author: OpenAI

OpenAI

This benchmark entry retains observations labeled o1 when the source does not distinguish the December 5, 2024 ChatGPT model from the newly post-trained December 17 API checkpoint. It asserts neither an exact checkpoint nor a release date; explicitly identified API-checkpoint observations are attached to the separate o1 model.

Benchmark scores

Benchmark	Score
Agentic Tool Use (Chat)	60.4
Agentic Tool Use (Enterprise)	70.1
BioTIER Refusal	64.6
BuzzBench	59.3
CharXiv Reasoning	52.6
CharXiv-D	87.5
DiploBench	91.0
EnigmaEval	5.7
Epoch Capabilities Index (ECI)	142
ERQA	53.3
FORTRESS Average Risk Score (ARS)	19.4
HPCT Refusal (CAIS)	31.0
Humanity's Last Exam (Preview)	8.8
Humanity's Last Exam (Text Only)	7.8
Humanity's Last Exam Diamond	3.0
Humanity's Last Exam RMS Calibration Error (CAIS)	83.0
Humanity's Last Exam Text Only (Preview)	8.3
IntPhys 2	53.1
MASK	59.3
MATH-Vision	60.3
METR Time Horizons (1.1, 50% Success)	38.8
MindCube Tiny	51.2
Pencil Puzzle Bench - Agentic	0.0
Pencil Puzzle Bench - Direct Ask	0.7
Public Goods Game Average Contribution	2.5
SpatialViz-Bench	41.4
VCT-Refusal (CAIS)	0.0
VISTA	45.3