BALROG — MiniHack (LLM)

Name: BALROG — MiniHack (LLM)
Published: 2024-11-20

Games · 2024-11-20

Mean per-episode progress (0-100) on MiniHack roguelike tasks (mazes, corridor battles, boxoban and quest levels) built on the NetHack engine, in BALROG's language-only track with a zero-shot agent. One of the six environments averaged into the BALROG leaderboard score; the episode count depends on the submission's protocol version.

Top models (higher is better)

Model	Score
GPT-6 Astra	65.0
GPT-5.6 Sol	42.5
Gemini 3 Pro Preview	40.0
Claude Opus 5	37.5
GPT-5.6 Terra	37.5
Gemini 3.1 Pro Preview	35.0
Opus 4.5	30.0
Gemini 3 Flash Preview	30.0
DeepSeek-R1	25.0
GPT-5.6 Luna	22.5
Grok 3 (API)	22.5
GPT-5	20.0
Gemini 2.5 Pro Experimental 03-25	17.5
Grok 4	17.5
Claude 3.5 Sonnet (Oct 2024)	15.0
Haiku 4.5	15.0
Gemini 2.5 Flash	12.5
Haiku 3.5	10.0
GPT-4o (2024-05-13)	10.0
GPT-4o Mini	10.0
Reka Flash 3	10.0
Llama 3.1 70B Instruct	7.5
Gemini 1.5 Flash 002	5.0
Gemini 1.5 Pro 002	5.0
Llama 3.1 8B Instruct	5.0
Llama 3.2 1B Instruct	5.0
Llama 3.2 90B Vision Instruct	5.0
Llama 3.3 70B Instruct	5.0
Phi-4	5.0
Qwen2.5 72B Instruct	5.0
DeepSeek-R1-Distill-Qwen-32B	2.5
Llama 3.2 11B Vision Instruct	2.5
Llama 3.2 3B Instruct	2.5
Mistral NeMo Instruct 2407	2.5
Qwen2 VL 72B Instruct	2.5
Qwen2 VL 7B Instruct	2.5
Qwen2.5 7B Instruct	0.0