Phi-3 Medium 128K Instruct
Microsoft Research · 2024-05-21 · 14B parameters
Phi-3 Medium 128K Instruct is Microsoft's open-weight, dense, instruction-tuned Phi-3 Medium checkpoint, publicly released on May 21, 2024. Its original 4,096-token RoPE window is extended to 131,072 tokens with LongRoPE, and Microsoft reports a rounded 4.8 trillion-token training run followed by supervised fine-tuning and direct preference optimization for instruction following, preference alignment, and safety. The model card gives only an October 2023 data cutoff, so the structured knowledge cutoff remains unknown at day precision.