Learn2Play Bench
Games · 2026-10-06
NUS's Learn2Play Bench tests whether LLM agents learn hidden rules from experience in 20 unfamiliar text games, ten repeating one instance and ten re-shuffling the visible instance across episodes. Agents play repeated episodes through a black-box command interface: 5 per game, or 10 for the five challenge games. Each episode's terminal score is normalized by a per-game reference to 0-100. Max is the mean over three independent trials of each trial's peak normalized score, averaged across games with challenge games weighted 3 and the others weighted 1. Max is the paper's primary ranking metric.
Top models (higher is better)
| Model | Score |
|---|---|
| Claude Opus 5 | 81.8 |
| Claude Opus 5.5 | 80.1 |
| GPT-6 Astra | 76.2 |
| GPT-5.6 Sol | 74.3 |
| Gemini 3.5 Flash | 72.0 |
| Kimi K3 | 68.5 |
| Gemini 3.7 Flash | 58.3 |
| Sonnet 5 | 55.9 |
| GPT-5 Mini | 34.0 |