Qwen 14B
Alibaba · 2023-09-25 · 14.2B parameters
Qwen 14B is Alibaba Cloud's pretrained base language model from the original Qwen family, released on September 25, 2023. The dense Transformer checkpoint was pretrained on more than 3 trillion tokens and serves as the base for Qwen-14B-Chat. Its 2,048-token training sequence length is extended to an 8,192-token maximum in the released configuration using dynamic NTK-aware interpolation and LogN attention scaling.
Benchmark scores
| Benchmark | Score |
|---|---|
| ARC AI2 | 84.4 |
| BBH (BIG-Bench Hard) | 53.4 |
| BoolQ | 86.2 |
| Capability | 103.9 |
| Epoch Capabilities Index (ECI) | 112 |
| LAMBADA | 71.1 |
| PIQA | 79.9 |