Smaug-Llama-3-70B-Instruct
Abacus.AI · 2024-05-17 · 70.6B parameters
Smaug-Llama-3-70B-Instruct is Abacus.AI's fine-tune of Meta's Llama 3 70B Instruct checkpoint, built with a new Smaug recipe intended to improve performance on real-world multi-turn conversations. Its model card reports a 9.21 MT-Bench average, compared with 9.19 for GPT-4-Turbo and 9.01 for the base checkpoint, and a 56.7 Arena-Hard score, compared with 41.1 for the base checkpoint.
Benchmark scores
| Benchmark | Score |
|---|---|
| EQ-Bench v2 | 80.7 |
| EQ-Bench v2 + MAGI-Hard Combined | 74.0 |
| MAGI-Hard | 67.3 |