OSWorld (self-reported, A11y tree)
Agents · 2024-04-11
Self-reported OSWorld success rate with accessibility-tree observation.
Top models (higher is better)
| Model | Score |
|---|---|
| Claude 3.5 Sonnet (June 2024) | 22.5 |
| GPT-4o | 16.6 |
| GPT-4 | 12.2 |
| GPT-4 Turbo | 10.8 |
| Gemini 1.5 Pro | 10.3 |
| Qwen Max (source-unspecified mainline) | 6.9 |
| GPT-4o Mini | 6.2 |
| Qwen VL Max 0809 | 4.3 |
| Mixtral 8x7B | 3.0 |
| LLaVA-OneVision 72B | 2.7 |
| Gemini 1.0 Pro | 2.4 |
| Llama 3 70B | 1.6 |