Qwen 14B
Alibaba · 2023-09-25 · 14.2B parameters
Qwen 14B is Alibaba Cloud's pretrained base language model from the original Qwen family, released on September 25, 2023. The dense Transformer checkpoint was pretrained on more than 3 trillion tokens and serves as the base for Qwen-14B-Chat. Its 2,048-token training sequence length is extended to an 8,192-token maximum in the released configuration using dynamic NTK-aware interpolation and LogN attention scaling.
Benchmark scores
| Benchmark | Score |
|---|---|
| Epoch Capabilities Index (ECI) | 113 |