Atlas

Benchmarks

← All benchmarks

Japanese

General QA · 2024-12-18

Elo rating on Scale SEAL's Japanese multilingual prompt set.

Top models (higher is better)

ModelScore
o1 Preview1118
Gemini Exp-12061117
o3-mini1104
Gemini 1.5 Pro Experimental 08271093
O11086
Gemini Experimental 11211084
GPT-4o (2024-08-06)1077
Claude 3.5 Sonnet (June 2024)1069
Gemini 2.0 Flash Experimental1064
GPT-4 Turbo (1106 Preview)978
Mistral Large 2 (Instruct 2407)967
Gemini 1.5 Flash Preview 0514966
Aya Expanse 32B960
Llama 3.1 405B Instruct895
Gemma 2 27B892
Llama 3.3 70B Instruct778
Aya-23-35B760
Loading Atlas data…