SOLO Bench Medium
Chat & Writing · 2025-05-01
SOLO Bench Medium increases the SOLO Bench task to 500 unique constrained sentences. It is substantially harder because the model must maintain more long-output constraints without repeating list words.
Top models (higher is better)
| Model | Score |
|---|---|
| Gemini 2.5 Pro Preview 03-25 | 57.8 |
| Claude 3.7 Sonnet | 13.6 |
| DeepSeek-R1 | 11.8 |
| o3 | 8.2 |
| GPT-4.5 | 5.8 |
| Grok 3 | 3.8 |