MMLU (uncertainty-routed CoT@32)
General QA · 2023-12-19
MMLU test-split accuracy with Google's uncertainty-routed chain of thought: 32 chain-of-thought samples, the majority answer is used when consensus exceeds a threshold tuned on the validation split, and otherwise the greedy answer without chain of thought. This multi-sample routing protocol is kept separate from single-sample MMLU.
Top models (higher is better)
| Model | Score |
|---|---|
| Gemini 1.0 Ultra | 90.0 |
| GPT-4 (source unspecified) | 87.3 |