Atlas

Benchmarks

← All benchmarks

BALROG — MiniHack (VLM)

Games · 2024-11-20

Mean per-episode progress (0-100) on MiniHack roguelike tasks (mazes, corridor battles, boxoban and quest levels) built on the NetHack engine, in BALROG's vision-language track, where the zero-shot agent also sees the current rendered frame. One of the five environments averaged into the BALROG VLM score.

Top models (higher is better)

ModelScore
Claude 3.5 Sonnet (Oct 2024)22.5
Gemini 2.5 Pro Experimental 03-2512.5
Gemini 1.5 Pro 0025.0
GPT-4o (2024-05-13)5.0
Qwen2 VL 7B Instruct5.0
Gemini 1.5 Flash 0022.5
GPT-4o Mini2.5
Llama 3.2 11B Vision Instruct2.5
Llama 3.2 90B Vision Instruct2.5
Qwen2 VL 72B Instruct2.5
Loading Atlas data…