Atlas

Models

← All models

StarCoder2-15B

Hugging Face · 2024-02-28 · 16B parameters

StarCoder2-15B is an open-weight, 15B-class code language model from BigCode, trained by NVIDIA within the collaboration jointly led by Hugging Face and ServiceNow. Its technical report describes an approximately 4.1-trillion-token base stage followed by an approximately 200-billion-token long-context stage on a 600-plus-programming-language corpus built from The Stack v2 and supplemental sources, using fill-in-the-middle training, grouped-query attention, and 4,096-token sliding-window attention within a 16,384-token context. The report found that it matched or outperformed CodeLlama-34B on several evaluations despite being less than half its size.

Benchmark scores

BenchmarkScore
ARC AI247.2
BBH (Open LLM Leaderboard v2)44.5
BigCodeBench Complete38.4
BigCodeBench-Hard Complete12.2
Capability99.9
Epoch Capabilities Index (ECI)104
GPQA (Open LLM Leaderboard v2)27.3
GSM8K57.7
IFEval (Open LLM Leaderboard v2)27.8
MATH Level 5 (Open LLM Leaderboard v2)6.0
MMLU64.1
MMLU-Pro (Open LLM Leaderboard v2)23.5
MuSR (Open LLM Leaderboard v2)35.0
WinoGrande64.3
Loading Atlas data…