DeepSeek-V3
DeepSeek · 2024-12-26 · 684.5B parameters
DeepSeek-V3 is DeepSeek's open-weight Mixture-of-Experts chat model, released December 26, 2024 and marketed as a 671B-parameter main model with 37B parameters activated per token; its released checkpoint also includes one multi-token-prediction module. The base model was pretrained on 14.8 trillion tokens, extended from 4K to 128K context with YaRN, and then aligned through supervised fine-tuning and GRPO reinforcement learning with reasoning patterns distilled from DeepSeek-R1.