Atlas

Models

← All models

Qwen2.5-Coder-7B-Instruct

Alibaba · 2024-09-19 · 7.6B parameters

Qwen2.5-Coder-7B-Instruct is Alibaba's instruction-tuned 7B code model, released September 19, 2024. Qwen reports that the Coder family was continued-pretrained on over 5.5 trillion tokens, with its public-repository source-code component spanning 92 programming languages; this checkpoint was then post-trained for coding assistance. The released configuration is set to 32,768 tokens and supports extrapolation to 131,072 tokens with YaRN for long-context use.

Benchmark scores

BenchmarkScore
Artificial Analysis Intelligence Index4.5
BBH (Open LLM Leaderboard v2)50.1
BigCodeBench Complete48.8
BigCodeBench Instruct40.4
BigCodeBench-Hard Complete20.3
BigCodeBench-Hard Instruct20.3
Capability111.9
GPQA (Open LLM Leaderboard v2)29.2
GPQA Diamond33.9
GSM8K86.7
Humanity's Last Exam (Text-Only)4.8
IFEval (Open LLM Leaderboard v2)61.0
LiveCodeBench (Artificial Analysis)12.6
MATH Level 5 (Open LLM Leaderboard v2)37.2
MATH-500 (Artificial Analysis source)66.0
MMLU Pro47.3
MMLU-Pro (Open LLM Leaderboard v2)33.5
MuSR (Open LLM Leaderboard v2)40.7
SciCode (Artificial Analysis)14.8
SnakeBench Average Apples0.2
SnakeBench Best Apples1.0
SnakeBench Rating17.5
SnakeBench Total Apples11.0
SnakeBench Win Rate39.5
Loading Atlas data…