Qwen 7B
Alibaba · 2023-08-03 · 7.7B parameters
Qwen-7B is Alibaba Cloud's dense base language model from the original Qwen generation, first released on August 3, 2023 and updated on September 25 to a stronger checkpoint pretrained on more than 2.4 trillion tokens; it serves as the base for Qwen-7B-Chat. The released configuration records an 8,192-token sequence length and a 32,768-token maximum with dynamic NTK and LogN attention scaling enabled; the architecture uses RoPE, SwiGLU, and RMSNorm.
Benchmark scores
| Benchmark | Score |
|---|---|
| ARC AI2 | 75.3 |
| BBH (BIG-Bench Hard) | 45.0 |
| BoolQ | 76.4 |
| Capability | 94.5 |
| Epoch Capabilities Index (ECI) | 106 |
| GSM8K | 51.7 |
| LAMBADA | 67.9 |
| MMLU | 45.0 |
| PIQA | 77.9 |