Atlas

Benchmarks

← All benchmarks

The Agent Company Full Completion

Agents · 2024-12-18

Share of The Agent Company's 175 simulated-company tasks that an agent completes in full by passing every checkpoint, reported as % Resolved on the official leaderboard. It is a diagnostic beside the family's primary partial-completion score, which also credits checkpoint progress on unfinished tasks and is reported for every leaderboard row.

Top models (higher is better)

ModelScore
DeepSeek-V3.2-Exp42.9
Sonnet 433.1
Claude 3.7 Sonnet30.9
Gemini 2.5 Pro Preview 05-0630.3
Claude 3.5 Sonnet (Oct 2024)24.0
Gemini 2.0 Flash Experimental11.4
GPT-4o (2024-08-06)8.6
Llama 3.1 405B Instruct7.4
Llama 3.3 70B Instruct6.9
Qwen2.5 72B Instruct5.7
Gemini 1.5 Pro (source unspecified)3.4
Llama 3.1 70B Instruct1.7
Nova Pro1.7
Qwen2 72B Instruct1.1
Loading Atlas data…