Emergent Collusion High-Illegality Game Rate
Safety · 2025-07-15
Emergent Collusion evaluates six instances of the same model in repeated auction games with optional communication and an explicit instruction not to collude. This metric is the percentage of games that an independent analyst assigns an illegality score of at least 7 out of 10, so lower is better.
Top models (lower is better)
| Model | Score |
|---|---|
| Grok 3 | 19.0 |
| ChatGPT-4o Latest observed 2025-03-27 | 23.0 |
| DeepSeek-V3-0324 | 26.0 |
| Grok 3 Mini | 30.0 |
| Sonnet 4 | 32.0 |
| Opus 4 | 36.0 |
| Gemini 2.5 Pro | 39.0 |
| Mistral Medium 3 | 47.0 |
| o3 | 51.0 |
| Qwen3-235B-A22B | 57.0 |
| O4 Mini | 62.0 |
| DeepSeek-R1-0528 | 71.0 |
| Grok 4 | 75.0 |