Atlas

Benchmarks

← All benchmarks

BALROG — BabyAI (LLM)

Games · 2024-11-20

Mean per-episode progress (0-100) on BabyAI grid-world instruction following: 2D navigation and object-manipulation missions described in text, in BALROG's language-only track with a zero-shot agent. One of the six environments averaged into the BALROG leaderboard score; the episode count depends on the submission's protocol version.

Top models (higher is better)

ModelScore
Claude Opus 5100.0
Gemini 3.1 Pro Preview100.0
GPT-6 Astra100.0
GPT-5.6 Luna98.0
GPT-5.6 Sol98.0
GPT-5.6 Terra98.0
Gemini 3 Pro Preview96.0
Gemini 3 Flash Preview86.0
Opus 4.580.0
Gemini 2.5 Pro Experimental 03-2580.0
GPT-580.0
GPT-4o (2024-05-13)77.6
Grok 476.0
Reka Flash 376.0
DeepSeek-R174.0
Llama 3.1 70B Instruct73.2
Llama 3.2 90B Vision Instruct72.0
Claude 3.5 Sonnet (Oct 2024)68.0
Gemini 2.5 Flash68.0
Haiku 4.566.0
Llama 3.3 70B Instruct66.0
Grok 3 (API)62.0
Gemini 1.5 Pro 00258.4
Haiku 3.552.0
GPT-4o Mini50.4
Gemini 1.5 Flash 00250.0
Llama 3.2 11B Vision Instruct50.0
Mistral NeMo Instruct 240750.0
DeepSeek-R1-Distill-Qwen-32B48.0
Llama 3.1 8B Instruct36.0
Qwen2.5 72B Instruct34.0
Phi-432.0
Qwen2 VL 72B Instruct24.0
Llama 3.2 3B Instruct20.0
Qwen2.5 7B Instruct14.0
Llama 3.2 1B Instruct8.0
Qwen2 VL 7B Instruct4.0
Loading Atlas data…