DeepSeek LLM 67B Chat
DeepSeek · 2023-11-29 · 67.4B parameters
DeepSeek LLM 67B Chat is the instruction-tuned checkpoint initialized from DeepSeek LLM 67B Base and fine-tuned on extra instruction data; DeepSeek reports around 1.5 million English and Chinese SFT instances and two SFT epochs for the 67B model. The base model was pretrained from scratch on a corpus reported as 2 trillion primarily English and Chinese tokens, and the released chat checkpoint supports a 4,096-token sequence length.