Qwen 7B
Alibaba · 2023-08-03 · 7.7B parameters
Qwen-7B is Alibaba Cloud's dense base language model from the original Qwen generation, first released on August 3, 2023 and updated on September 25 to a stronger checkpoint pretrained on more than 2.4 trillion tokens; it serves as the base for Qwen-7B-Chat. The released configuration records an 8,192-token sequence length and a 32,768-token maximum with dynamic NTK and LogN attention scaling enabled; the architecture uses RoPE, SwiGLU, and RMSNorm.
Benchmark scores
| Benchmark | Score |
|---|---|
| Epoch Capabilities Index (ECI) | 106 |