Atlas

Benchmarks

← All benchmarks

BALROG — BabaIsAI (VLM)

Games · 2024-11-20

Mean per-episode progress (0-100) on Baba Is AI rule-manipulation puzzles adapted from Baba Is You, where agents push word blocks to rewrite the level's rules, in BALROG's vision-language track, where the zero-shot agent also sees the current rendered frame. One of the five environments averaged into the BALROG VLM score.

Top models (higher is better)

ModelScore
Gemini 2.5 Pro Experimental 03-2553.3
Claude 3.5 Sonnet (Oct 2024)34.5
Gemini 1.5 Pro 00231.4
Llama 3.2 90B Vision Instruct21.9
GPT-4o (2024-05-13)18.6
GPT-4o Mini16.4
Qwen2 VL 7B Instruct11.9
Gemini 1.5 Flash 0028.3
Qwen2 VL 72B Instruct5.9
Llama 3.2 11B Vision Instruct5.8
Loading Atlas data…