Atlas

Models

← All models

StarCoder2-3B

Hugging Face · 2024-02-28 · 3B parameters

StarCoder2-3B is an open-weight code language model from the BigCode Project, trained by ServiceNow as the smallest member of the 3B, 7B, and 15B StarCoder2 family. Its report describes an approximately 3.1-trillion-token base stage using 4,096-token sequences followed by an approximately 200-billion-token long-context stage, with fill-in-the-middle training, grouped-query attention, and 4,096-token sliding-window attention within a 16,384-token context. The authors report that it outperforms comparable small code models on most evaluated benchmarks and can match or surpass StarCoderBase-15B.

Benchmark scores

BenchmarkScore
ARC AI234.2
BBH (Open LLM Leaderboard v2)35.1
BigCodeBench Complete21.4
BigCodeBench-Hard Complete4.7
Capability80.9
Epoch Capabilities Index (ECI)87.0
GPQA (Open LLM Leaderboard v2)24.4
GSM8K21.6
IFEval (Open LLM Leaderboard v2)20.4
MATH Level 5 (Open LLM Leaderboard v2)1.5
MMLU36.6
MMLU-Pro (Open LLM Leaderboard v2)16.4
MuSR (Open LLM Leaderboard v2)34.3
WinoGrande57.1
Loading Atlas data…