Qwen 1.8B
Alibaba · 2023-11-30 · 1.8B parameters
Qwen-1.8B is the 1.8B dense pretrained base model from the original Qwen generation, released by Alibaba Cloud on November 30, 2023, with a corresponding aligned Qwen-1.8B-Chat variant. Its released configuration records an 8,192-token sequence/position length and enables dynamic NTK plus LogN attention scaling, while Qwen documents a 32,768-token extrapolated maximum.
Benchmark scores
| Benchmark | Score |
|---|---|
| ARC AI2 | 53.2 |
| BBH (BIG-Bench Hard) | 28.2 |
| BoolQ | 68.0 |
| Capability | 75.6 |
| Epoch Capabilities Index (ECI) | 91.3 |
| GSM8K | 21.2 |
| LAMBADA | 58.4 |
| MMLU | 28.2 |
| PIQA | 73.3 |