LLM Position Bias Order Flip Rate
Chat & Writing · 2026-04-21
Position-bias diagnostic metric: LLM Position Bias Order Flip Rate. The benchmark uses paired story comparisons under order swaps.
Top models (lower is better)
| Model | Score |
|---|---|
| MiMo-V2-Pro | 19.8 |
| Claude Fable 5 | 27.3 |
| Opus 4.8 | 27.4 |
| Doubao Seed 2.0 Pro | 28.0 |
| Gemini 3.5 Flash | 29.8 |
| Opus 4.6 | 30.2 |
| DeepSeek-V3.2 | 30.3 |
| GLM-5.1 | 31.5 |
| Qwen3.6 Plus (2026-04-02) | 33.8 |
| Qwen3.7-Max | 34.8 |
| MiniMax M3 | 34.9 |
| Gemini 3.1 Pro Preview | 35.4 |
| MiniMax M2.7 | 36.5 |
| Trinity Large Thinking | 36.6 |
| Sonnet 4.6 | 37.4 |
| Opus 4.7 | 37.7 |
| Grok 4.20 | 39.0 |
| DeepSeek-V4-Pro | 43.6 |
| GPT-5.5 | 44.4 |
| ERNIE 5.0 | 45.0 |
| Gemini 3.1 Flash-Lite Preview | 45.3 |
| Kimi K2.6 | 47.3 |
| Qwen3.5 397B A17B | 48.5 |
| Gemma 4 31B IT | 50.8 |
| Kimi K2.5 | 51.6 |
| Qwen3.5 122B A10B | 51.7 |
| GPT-5.4 Mini | 54.7 |
| GPT-5.4 | 57.4 |
| Mistral Medium 3.1 | 57.9 |
| Mistral Large 3 675B Instruct 2512 | 58.7 |
| Mistral Medium 3.5 | 72.5 |