Confabulations Leaderboard Weighted Score
Safety · 2024-10-10
Weighted score combining confabulation and non-response.
Top models (lower is better)
| Model | Score |
|---|---|
| GPT-5 | 10.3 |
| Gemini 2.5 Pro Preview 05-06 | 10.6 |
| Grok 3 Mini | 10.8 |
| Gemini 2.5 Pro Preview 03-25 | 10.8 |
| GLM-4.5 | 11.3 |
| O1 | 11.7 |
| Qwen3-30B-A3B | 12.3 |
| Gemini 2.5 Pro | 12.4 |
| Grok 4 | 12.4 |
| Gemini 2.0 Flash Thinking Experimental 01-21 | 12.4 |
| DeepSeek-R1 | 12.7 |
| o1 Preview | 13.0 |
| Sonnet 4 | 13.2 |
| GPT-5 Mini | 13.3 |
| Gemini 1.5 Pro 002 | 13.5 |
| GPT-4.5 | 13.6 |
| Grok 3 | 14.2 |
| o3-pro | 14.2 |
| o3 | 14.4 |
| DeepSeek-R1-0528 | 14.6 |
| Claude 3.7 Sonnet | 14.7 |
| GPT-4o (2024-08-06) | 15.3 |
| QwQ-32B | 15.6 |
| Qwen3-235B-A22B | 15.6 |
| gpt-oss-120b | 15.7 |
| O4 Mini | 15.8 |
| Opus 4 | 15.9 |
| ERNIE 4.5 300B A47B | 16.0 |
| ChatGPT-4o Latest observed 2025-01-29 | 16.6 |
| Qwen3 235B A22B Thinking 2507 | 16.8 |
| Gemini 2.5 Flash Preview 04-17 | 16.8 |
| Opus 4.1 | 17.1 |
| GPT-4o (2024-11-20) | 17.2 |
| Llama 3.1 405B | 17.6 |
| o3-mini | 17.9 |
| Gemini 2.0 Pro Experimental 02-05 | 18.4 |
| o1-mini | 18.6 |
| Qwen2.5 72B Instruct | 19.1 |
| Claude 3.5 Sonnet (Oct 2024) | 19.9 |
| Grok 2 | 20.1 |