Atlas

Models

← All models

Tülu 2 DPO 70B

Allen Institute for AI (Ai2) · 2023-11-17 · 69B parameters

Tülu 2 DPO 70B is Ai2's instruction-following Llama 2 70B checkpoint, first supervised-fine-tuned on the Tülu V2 mixture and then further aligned with Direct Preference Optimization on a filtered, binarized version of UltraFeedback. Ai2 publicly released the checkpoint, data, and training and evaluation code in November 2023.

Benchmark scores

BenchmarkScore
AlpacaEval 2.0 (length-controlled win rate)21.2
AlpacaEval 2.0 (raw win rate)16.0
Arena Text — Business, Management & Financial Ops (No Style Control)1077
Arena Text — Chinese (No Style Control)1013
Arena Text — Creative Writing (No Style Control)1105
Arena Text — English (No Style Control)1150
Arena Text — Entertainment, Sports & Media (No Style Control)1116
Arena Text — Exclude Ties (No Style Control)963
Arena Text — Exclude Ties (Style Controlled)1043
Arena Text — Hard Prompts (No Style Control)1105
Arena Text — Hard Prompts English (No Style Control)1126
Arena Text — Instruction Following (No Style Control)1111
Arena Text — Legal & Government (No Style Control)1135
Arena Text — Life, Physical & Social Science (No Style Control)1105
Arena Text — Longer Query (No Style Control)1106
Arena Text — Medicine & Healthcare (No Style Control)1059
Arena Text — Multi-turn (No Style Control)1085
Arena Text — Non-English (No Style Control)1068
Arena Text — Non-English (Style Controlled)1125
Arena Text — Occupational Mathematical (No Style Control)1099
Arena Text — Software & IT Services (No Style Control)1111
Arena Text — Writing, Literature & Language (No Style Control)1123
Capability110.7
EQ-Bench v276.6
EQ-Bench v2 + MAGI-Hard Combined63.4
LMArena Text Arena Chinese Elo1103
LMArena Text Arena Coding Elo1214
LMArena Text Arena Creative Writing Elo1146
LMArena Text Arena Elo1177
LMArena Text Arena English Elo1205
Loading Atlas data…