Baichuan2-13B-Base
Baichuan · 2023-09-06 · 13.9B parameters
Baichuan2-13B-Base is the 13B base checkpoint in Baichuan Intelligence's second-generation open-weight model family, released in September 2023 and pretrained from scratch on 2.6 trillion tokens. It is a multilingual decoder-only Transformer with a 4,096-token sequence length and uses ALiBi positional encoding, SwiGLU activations, and RMSNorm. It was released alongside Baichuan2-7B-Base and the corresponding 7B and 13B Chat models.
Benchmark scores
| Benchmark | Score |
|---|---|
| ARC AI2 | 38.0 |
| BoolQ | 67.0 |
| Capability | 96.5 |
| Epoch Capabilities Index (ECI) | 102 |
| HellaSwag (Unspecified Scoring Protocol) | 70.8 |
| LAMBADA | 74.0 |
| PIQA | 78.1 |