WizardLM-2 8x22B
Microsoft Research · 2024-04-15
WizardLM-2 8x22B is Microsoft's most advanced model in the April 15, 2024 WizardLM-2 release, an open-weight mixture-of-experts instruction model based on Mixtral 8x22B. The family was post-trained with a fully AI-powered synthetic-data pipeline using progressive Evol-Instruct and Evol-Answer generation, supervised learning, staged DPO, and RLEIF for complex chat, multilingual, reasoning, and agent tasks. Microsoft's released model card described it as 141B parameters, while Mistral described the base as 39B active; these are rounded disclosures, so the exact WizardLM checkpoint counts remain unknown.
Benchmark scores
| Benchmark | Score |
|---|---|
| Capability | 116.5 |
| Creative Writing v2 | 78.9 |
| EQ-Bench v2 | 77.9 |
| EQ-Bench v2 + MAGI-Hard Combined | 68.5 |
| GPQA Diamond | 43.4 |
| MAGI-Hard | 59.2 |
| MATH Level 5 | 25.7 |
| SnakeBench Average Apples | 1.2 |
| SnakeBench Best Apples | 4.0 |
| SnakeBench Rating | 16.0 |
| SnakeBench Total Apples | 28.0 |
| SnakeBench Win Rate | 38.9 |