Atlas

Benchmarks

← All benchmarks

BALROG — BabyAI (VLM)

Games · 2024-11-20

Mean per-episode progress (0-100) on BabyAI grid-world instruction following: 2D navigation and object-manipulation missions described in text, in BALROG's vision-language track, where the zero-shot agent also sees the current rendered frame. One of the five environments averaged into the BALROG VLM score.

Top models (higher is better)

ModelScore
Claude 3.5 Sonnet (Oct 2024)82.0
Gemini 2.5 Pro Experimental 03-2574.0
Llama 3.2 90B Vision Instruct66.0
GPT-4o (2024-05-13)62.0
Gemini 1.5 Pro 00258.4
Gemini 1.5 Flash 00243.2
GPT-4o Mini38.0
Qwen2 VL 72B Instruct34.0
Llama 3.2 11B Vision Instruct18.0
Qwen2 VL 7B Instruct2.0
Loading Atlas data…