StarCoder2-7B
Hugging Face · 2024-02-28 · 7.2B parameters
StarCoder2-7B is the Hugging Face-trained medium member of BigCode’s open code-model family. The technical report describes an approximately 3.5-trillion-token base stage followed by an approximately 200-billion-token long-context stage on the StarCoder2 pretraining corpus, using fill-in-the-middle training, grouped-query attention, and 4,096-token sliding-window attention within a 16,384-token context. It is a base model for code completion and infilling, not an instruction-tuned assistant.