LisanBench Difficulty-Weighted
Games · 2025-05-30
LisanBench Difficulty-Weighted Score sums valid word-chain moves across 50 starting words after weighting each move by inverse branching factor and word rarity, then averaging the three trials for each word. It rewards paths through sparse, uncommon parts of the word graph rather than raw chain length alone.
Top models (higher is better)
| Model | Score |
|---|---|
| Opus 4.7 | 5123 |
| Claude Opus 5 | 5068 |
| Claude Fable 5 | 4562 |
| Opus 4.6 | 3526 |
| Kimi K3 | 3526 |
| GPT-5.5 | 3316 |
| Sonnet 4.6 | 2944 |
| GPT-5.4 | 2738 |
| Opus 4.8 | 2694 |
| Opus 4.5 | 2204 |
| Gemini 3.1 Pro Preview | 1929 |
| Grok 4 | 1778 |
| Sonnet 5 | 1737 |
| o3 | 1523 |
| DeepSeek-V3.2-Speciale | 1511 |
| Grok 4.20 | 1465 |
| GPT-5.2 | 1459 |
| GPT-5 | 1457 |
| GPT-5.6 Sol | 1403 |
| GPT-5.6 Terra | 1197 |
| Gemini 3 Pro Preview | 1131 |
| Gemini 3.5 Flash | 1128 |
| Sonnet 4.5 | 1091 |
| DeepSeek-V4-Flash | 1063 |
| DeepSeek-V4-Pro | 1060 |
| DeepSeek-V3.2 | 925 |
| Step 3.5 Flash | 811 |
| Grok 4 Fast | 806 |
| GPT-5 Mini | 759 |
| GPT-5.6 Luna | 648 |
| Kimi K2.5 | 642 |
| Kimi K2 Thinking | 633 |
| GPT-5 Nano | 627 |
| Grok 4.1 Fast | 604 |
| Sonnet 4 | 603 |
| GLM-5.2 | 592 |
| Gemini 3 Flash Preview | 592 |
| GPT-5.4 Mini | 591 |
| GPT-5.4 Nano | 543 |
| o3-mini | 518 |