Atlas

Benchmarks

← All benchmarks

BALROG — NetHack (VLM)

Games · 2024-11-20

Mean per-episode progress (0-100) on the full NetHack Learning Environment, scored by BALROG's dungeon-depth and experience-level progression metric, in BALROG's vision-language track, where the zero-shot agent also sees the current rendered frame. One of the five environments averaged into the BALROG VLM score.

Top models (higher is better)

ModelScore
Gemini 2.5 Pro Experimental 03-251.6
Claude 3.5 Sonnet (Oct 2024)1.2
Gemini 1.5 Pro 0020.5
GPT-4o (2024-05-13)0.4
Gemini 1.5 Flash 0020.0
GPT-4o Mini0.0
Llama 3.2 11B Vision Instruct0.0
Llama 3.2 90B Vision Instruct0.0
Qwen2 VL 72B Instruct0.0
Qwen2 VL 7B Instruct0.0
Loading Atlas data…