CulturalRiddles
General QA · 2026-10-21
Cohere Labs Community benchmark of riddles written by native speakers of 62 language-country communities, each answered once in a single prompt that names the country and topic. The score is the share of scored responses rated fully correct over the 5,690 riddles of the 57 communities with native-speaker review, after rule-based exact matching, native-speaker grading and a two-judge LLM jury that keeps a verdict only when the judges agree; responses that remain unscored are excluded.
Top models (higher is better)
| Model | Score |
|---|---|
| Gemini 3.1 Pro Preview | 79.4 |
| GPT-5.5 | 76.3 |
| Opus 4.7 | 66.0 |
| Gemini 2.5 Flash | 50.1 |
| Mistral Large 3 675B Instruct 2512 | 38.5 |
| Gemini 2.5 Flash-Lite | 33.6 |
| Qwen3.5-27B | 28.0 |
| Apertus 70B Instruct | 15.2 |
| Nova Lite | 14.7 |
| Olmo 3 32B Think | 11.3 |
| Nova Micro | 8.3 |
| Apertus 8B Instruct | 7.6 |
| Tiny Aya Global | 7.0 |
| Qwen3.5-0.8B | 0.4 |