Atlas

Benchmarks

← All benchmarks

BALROG — Crafter (VLM)

Games · 2024-11-20

Mean per-episode progress (0-100) on Crafter open-world survival: progress is the share of the game's 22 achievements unlocked, in BALROG's vision-language track, where the zero-shot agent also sees the current rendered frame. One of the five environments averaged into the BALROG VLM score.

Top models (higher is better)

ModelScore
Gemini 2.5 Pro Experimental 03-2537.3
Claude 3.5 Sonnet (Oct 2024)37.3
Gemini 1.5 Pro 00233.5
GPT-4o (2024-05-13)26.8
Gemini 1.5 Flash 00220.7
GPT-4o Mini19.9
Qwen2 VL 72B Instruct18.6
Llama 3.2 11B Vision Instruct15.9
Llama 3.2 90B Vision Instruct14.5
Qwen2 VL 7B Instruct5.5
Loading Atlas data…