DeepSeek-Coder-V2-Lite-Instruct
DeepSeek · 2024-06-14 · 15.7B parameters
DeepSeek-Coder-V2-Lite-Instruct is the instruction-aligned 16B-class MoE checkpoint released June 14, 2024; its downloadable weights contain exactly 15,706,484,224 parameters, while DeepSeek reports only the rounded figure of 2.4B active parameters. The family was continued-pretrained from an intermediate DeepSeek-V2 checkpoint on 6T additional tokens and this checkpoint was aligned with supervised fine-tuning and GRPO; it supports 338 programming languages, and DeepSeek documents 128K context support even though the YaRN configuration declares 163,840 maximum positions.