Vending-Bench
Games · 2025-02-20
Vending-Bench is a long-horizon business simulation where agents operate a vending-machine business over many turns; reported metrics include final valuation and units sold.
Top models (higher is better)
| Model | Score |
|---|---|
| Grok 4 | 4694 |
| Gemini 3 Pro Preview | 4388 |
| GPT-5 | 3579 |
| Sonnet 4.5 | 2465 |
| GPT-5.1 | 2380 |
| Claude 3.5 Sonnet (Oct 2024) | 2218 |
| Opus 4 | 2077 |
| o3 | 1843 |
| Claude 3.7 Sonnet | 1568 |
| Kimi K2 Thinking | 1296 |
| Gemini Robotics-ER 1.5 | 976 |
| Sonnet 4 | 968 |
| o3-mini | 907 |
| Human (Vending-Bench) | 844 |
| Gemini 2.5 Pro Preview 03-25 | 789 |
| Qwen3-235B-A22B | 616 |
| gpt-oss-120b | 608 |
| Gemini 1.5 Pro | 594 |
| GPT-4o Mini | 582 |
| GPT-5 Mini | 581 |
| Gemini 1.5 Flash | 572 |
| Haiku 3.5 | 373 |
| Gemini 2.0 Flash | 338 |
| GPT-4o | 335 |
| Gemini 2.0 Pro Experimental | 274 |
| Llama 4 Maverick | 157 |