Atlas

Models

← All models

Smaug-72B-v0.1

Abacus.AI · 2024-02-02 · 72.3B parameters

Smaug-72B-v0.1 is Abacus.AI's 72B instruction-tuned model fine-tuned from the Qwen-72B-derived moreh/MoMo-72B-lora-1.8.7-DPO checkpoint using DPO-Positive (DPOP). DPOP addresses a standard-DPO failure mode in which preferred-completion likelihood can decrease on response pairs with low edit distance; at release, Smaug-72B-v0.1 ranked first on the Open LLM Leaderboard.

Benchmark scores

BenchmarkScore
BBH (Open LLM Leaderboard v2)60.0
Capability116.9
EQ-Bench v279.8
EQ-Bench v2 + MAGI-Hard Combined70.0
GPQA (Open LLM Leaderboard v2)32.4
IFEval (Open LLM Leaderboard v2)51.7
MAGI-Hard60.2
MATH Level 5 (Open LLM Leaderboard v2)19.1
MMLU-Pro (Open LLM Leaderboard v2)46.2
MuSR (Open LLM Leaderboard v2)44.7
Loading Atlas data…