Baichuan-7B
Baichuan · 2023-06-15 · 7B parameters
Baichuan-7B is Baichuan Intelligent Technology's open-weight bilingual Chinese-English pretrained base language model, released in June 2023 and trained on approximately 1.2 trillion tokens. It is a dense decoder-only Transformer with 7,000,559,616 parameters, a 4,096-token context window, 32 layers, RoPE, SwiGLU, RMSNorm, and multi-head attention. Its 64,000-token tokenizer uses SentencePiece BPE with UTF-8 byte fallback.
Benchmark scores
| Benchmark | Score |
|---|---|
| Epoch Capabilities Index (ECI) | 89.8 |