Atlas

Benchmarks

← All benchmarks

BALROG (VLM)

Games · 2024-11-20

BALROG's vision-language track: zero-shot agents receive the text observation plus the current rendered frame. The score is the unweighted mean progress over five environments (BabyAI, Crafter, Baba Is AI, MiniHack and NetHack); TextWorld has no images and is not part of this track.

Top models (higher is better)

ModelScore
Gemini 2.5 Pro Experimental 03-2535.7
Claude 3.5 Sonnet (Oct 2024)35.5
Gemini 1.5 Pro 00225.8
GPT-4o (2024-05-13)22.6
Llama 3.2 90B Vision Instruct21.0
GPT-4o Mini15.4
Gemini 1.5 Flash 00214.9
Qwen2 VL 72B Instruct12.2
Llama 3.2 11B Vision Instruct8.4
Loading Atlas data…