Atlas

Models

← All models

Phi-3 Medium 128K Instruct

Microsoft Research · 2024-05-21 · 14B parameters

Phi-3 Medium 128K Instruct is Microsoft's open-weight, dense, instruction-tuned Phi-3 Medium checkpoint, publicly released on May 21, 2024. Its original 4,096-token RoPE window is extended to 131,072 tokens with LongRoPE, and Microsoft reports a rounded 4.8 trillion-token training run followed by supervised fine-tuning and direct preference optimization for instruction following, preference alignment, and safety. The model card gives only an October 2023 data cutoff, so the structured knowledge cutoff remains unknown at day precision.

Benchmark scores

BenchmarkScore
Adversarial NLI55.8
ARC AI291.6
BBH (BIG-Bench Hard)81.4
BBH (Open LLM Leaderboard v2)63.8
BigCodeBench Complete48.7
BigCodeBench Instruct37.6
BigCodeBench-Hard Complete15.5
BigCodeBench-Hard Instruct10.8
Capability117.3
GPQA (Open LLM Leaderboard v2)33.6
GPQA Diamond27.6
HellaSwag (Unspecified Scoring Protocol)82.4
IFEval (Open LLM Leaderboard v2)60.4
MATH Level 517.6
MATH Level 5 (Open LLM Leaderboard v2)19.2
MMLU-Pro (Open LLM Leaderboard v2)47.1
MuSR (Open LLM Leaderboard v2)41.3
OpenBookQA87.4
TriviaQA73.9
WinoGrande81.5
Loading Atlas data…