Atlas

Benchmarks

← All benchmarks

ZeroBench (Main questions, pass@1)

Multimodal · 2025-02-13

This ZeroBench split evaluates multimodal visual reasoning on the benchmark's main questions. Scores report pass@1 accuracy in percentage points.

Top models (higher is better)

ModelScore
Opus 4.15.0
GPT-5 Mini4.0
Opus 44.0
Sonnet 43.0
o33.0
Gemini 2.5 Pro Experimental 03-253.0
Gemini 2.5 Flash Preview 04-173.0
Claude 3.7 Sonnet3.0
GPT-5 Nano2.0
Seed1.5-VL2.0
O4 Mini2.0
GPT-51.0
Grok 41.0
GPT-4.51.0
Sonnet 4.50.0
Llama 4 Maverick Instruct0.0
Llama 4 Scout Instruct0.0
GPT-4.10.0
O1 Pro0.0
O10.0
Gemini 2.0 Flash Thinking Experimental 01-210.0
QVQ 72B Preview0.0
GPT-4o0.0
GPT-4o Mini0.0
Gemini 2.0 Flash0.0
Gemini 1.5 Pro0.0
Gemini 1.5 Flash0.0
Gemini 1.0 Pro Vision0.0
Claude 3.5 Sonnet (Oct 2024)0.0
Claude 3.5 Sonnet (June 2024)0.0
Opus 30.0
Claude 3 Sonnet0.0
Claude 3 Haiku0.0
Llama 3.2 90B Vision Instruct0.0
Qwen2 VL 72B Instruct0.0
NVLM-D-72B0.0
Pixtral Large0.0
Loading Atlas data…