MMLU Medical Genetics (Open Medical-LLM)
Science · 2024-04-19
The medical-genetics subset of MMLU, 100 questions, scored by the Open Medical-LLM Leaderboard under lm-evaluation-harness with a fixed prompt. At 100 items the binomial noise floor alone is about 4 percentage points, so small differences here are not meaningful.
Top models (higher is better)
| Model | Score |
|---|---|
| GPT-4 | 91.0 |
| Llama 3 8B | 83.0 |
| Llama 3 8B Instruct | 82.0 |
| Solar 10.7B Instruct V1.0 | 76.0 |
| GPT-3.5 Turbo 1106 | 74.0 |
| Mistral 7B v0.1 Base | 71.0 |
| Gemma 7B | 70.0 |
| Qwen1.5 7B | 69.0 |
| Qwen1.5 7B Chat | 67.0 |
| Zephyr 7B Beta | 66.0 |
| Gemma 1.1 7B IT | 66.0 |
| Mistral 7B Instruct v0.1 | 63.0 |
| Gemma 7B IT | 60.0 |
| Phi-1.5 | 42.0 |
| Falcon 7B | 30.0 |
| Gemma 2B | 28.0 |