Baichuan2-7B-Base
Baichuan · 2023-09-06 · 7.5B parameters
Baichuan2-7B-Base is the 7B base checkpoint in Baichuan Intelligence's second-generation open-weight model family, released in September 2023 and pretrained from scratch on 2.6 trillion tokens. It is a multilingual decoder-only Transformer with a 4,096-token sequence length and uses RoPE positional encoding, SwiGLU activations, and RMSNorm. It was released alongside Baichuan2-13B-Base and the corresponding 7B and 13B Chat models.
Benchmark scores
| Benchmark | Score |
|---|---|
| ARC AI2 | 32.5 |
| BoolQ | 63.2 |
| Capability | 90.4 |
| Epoch Capabilities Index (ECI) | 94.8 |
| HellaSwag (Unspecified Scoring Protocol) | 68.0 |
| LAMBADA | 73.3 |
| MMLU | 54.2 |