← All benchmarks

Okapi Multilingual MMLU

General QA · 2023-07-29

MMLU machine-translated with ChatGPT into 26 languages by the Okapi project (Lai et al., 2023), reported as accuracy averaged across languages. It is a different dataset from OpenAI's professionally translated MMMLU and is kept separate from it. Diagnostic: only the Claude 3 model card reports it in Atlas.

Top models (higher is better)