Atlas

Models

← All models

Zephyr 7B Alpha

Hugging Face · 2023-10-09 · 7.2B parameters

Zephyr 7B Alpha is the first assistant-tuned model in Hugging Face H4's Zephyr series, produced by distilled supervised fine-tuning of Mistral 7B v0.1 on a filtered UltraChat corpus generated by GPT-3.5 Turbo, followed by distilled direct preference optimization on GPT-4-ranked UltraFeedback preferences. Released as an open research artifact, it demonstrated that a 7B model post-trained with public synthetic alignment data could compete with much larger chat models, while the authors cautioned that the work focused on helpfulness rather than safety alignment.

Benchmark scores

BenchmarkScore
Arena Text — Creative Writing (No Style Control)1069
Arena Text — English (No Style Control)1092
Arena Text — Entertainment, Sports & Media (No Style Control)1049
Arena Text — Exclude Ties (No Style Control)876
Arena Text — Exclude Ties (Style Controlled)971
Arena Text — Hard Prompts (No Style Control)1024
Arena Text — Hard Prompts English (No Style Control)1044
Arena Text — Instruction Following (No Style Control)1035
Arena Text — Life, Physical & Social Science (No Style Control)1028
Arena Text — Multi-turn (No Style Control)1018
Arena Text — Non-English (No Style Control)988
Arena Text — Non-English (Style Controlled)1060
Arena Text — Software & IT Services (No Style Control)1054
Arena Text — Writing, Literature & Language (No Style Control)1082
BBH (Open LLM Leaderboard v2)45.8
Capability99.5
EQ-Bench v256.8
EQ-Bench v2 + MAGI-Hard Combined46.0
GPQA (Open LLM Leaderboard v2)29.8
IFEval (Open LLM Leaderboard v2)51.9
LMArena Text Arena Coding Elo1166
LMArena Text Arena Creative Writing Elo1100
LMArena Text Arena Elo1126
LMArena Text Arena English Elo1158
LMArena Text Arena Hard Prompts Elo1122
LMArena Text Arena Hard Prompts English Elo1137
LMArena Text Arena Industry: Writing, Literature & Language Elo1122
LMArena Text Arena Instruction Following Elo1098
LMArena Text Arena Multi-turn Elo1074
LMArena Text Arena Occupational Entertainment, Sports, and Media Elo1100
Loading Atlas data…