GSM1k leaderboard — Benchmark Atlas

← All benchmarks

GSM1k

Math

Scale's held-out GSM8K-style mathematics benchmark of 1,205 problems, evaluated with a five-shot prompt.

Top models (higher is better)

Model	Score
Claude 3.5 Sonnet (June 2024)	96.6
GPT-4o (2024-08-06)	95.7
Llama 3.1 405B Instruct	95.6
Opus 3	95.2
GPT-4 0125 Preview	95.1
GPT-4o	94.8
Gemini 1.5 Pro Experimental 0827	94.7
Mistral Large 2 (Instruct 2407)	93.9
Claude 3 Sonnet	93.3
Gemini 1.5 Pro Preview 0514	92.3
Gemini 1.5 Pro Preview 0409	90.5
Gemini 1.5 Flash Preview 0514	90.1
Llama 3 70B Instruct	90.1
Mistral Large 1.0	87.5
Gemini 1.0 Pro	79.8
Code Llama Instruct 34B	37.5