Atlas

Benchmarks

← All benchmarks

The Agent Company

Agents · 2024-12-18

A benchmark for LLM agents performing consequential real-world professional tasks in a simulated software company environment using web, code, programs, and coworker communication.

Top models (higher is better)

ModelScore
DeepSeek-V3.2-Exp52.4
Sonnet 443.2
Claude 3.7 Sonnet40.2
Gemini 2.5 Pro Preview 05-0639.3
Claude 3.5 Sonnet (Oct 2024)34.4
Gemini 2.0 Flash Experimental19.0
GPT-4o (2024-08-06)16.7
Llama 3.1 405B Instruct14.1
Llama 3.3 70B Instruct12.8
Qwen2.5 72B Instruct11.8
Gemini 1.5 Pro (source unspecified)8.0
Llama 3.1 70B Instruct6.5
Nova Pro5.7
Qwen2 72B Instruct4.2
Loading Atlas data…