StarCoder2-15B
Hugging Face · 2024-02-28 · 16B parameters
StarCoder2-15B is an open-weight, 15B-class code language model from BigCode, trained by NVIDIA within the collaboration jointly led by Hugging Face and ServiceNow. Its technical report describes an approximately 4.1-trillion-token base stage followed by an approximately 200-billion-token long-context stage on a 600-plus-programming-language corpus built from The Stack v2 and supplemental sources, using fill-in-the-middle training, grouped-query attention, and 4,096-token sliding-window attention within a 16,384-token context. The report found that it matched or outperformed CodeLlama-34B on several evaluations despite being less than half its size.