BoolQ
Classic NLP · 2019-05-24
A yes/no reading comprehension benchmark where models answer naturally occurring questions given a short supporting passage.
Top models (higher is better)
| Model | Score |
|---|---|
| PaLM 2-L | 90.9 |
| T5 3B | 89.9 |
| Inflection-1 | 89.7 |
| Stable Beluga 2 | 89.4 |
| GPT-4o Mini | 88.7 |
| PaLM 2-M | 88.6 |
| PaLM 2-S | 88.1 |
| InternLM 20B | 87.5 |
| GPT-3.5 Turbo 0613 | 87.0 |
| Qwen 14B | 86.2 |
| Gemini 1.5 Flash | 85.8 |
| Gemma 2 9B | 85.7 |
| PaLM 62B | 84.8 |
| Phi-3.5-MoE-instruct | 84.6 |
| Chinchilla | 83.7 |
| Llama 2 34B | 83.7 |
| Vicuna 13B v1.1 | 83.5 |
| Gemma 7B | 83.2 |
| Mistral 7B Instruct v0.2 | 83.2 |
| Llama 3.1 8B Instruct | 82.8 |
| Mistral NeMo Base 2407 | 82.5 |
| Vicuna 13B v1.3 | 80.8 |
| OPT-175B | 79.3 |
| Gemini Nano 2 | 79.3 |
| ChatGLM2 6B (chat) | 79.0 |
| GLM-130B | 78.4 |
| Phi-3.5 Mini Instruct | 78.0 |
| Qwen 7B | 76.4 |
| Phi-1.5 | 75.8 |
| XGen-7B-8K-Base | 74.3 |
| GPT-3.5 Turbo 0301 | 74.0 |
| GPT-3 Davinci | 72.2 |
| Gemini Nano 1 | 71.6 |
| OpenLLaMA 7B | 70.6 |
| Gemma 2B | 69.4 |
| RedPajama INCITE 7B Base | 69.3 |
| Qwen 1.8B | 68.0 |
| Baichuan2-13B-Base | 67.0 |
| GPT-3 Curie | 65.6 |
| InternLM 7B | 64.1 |