InternLM 7B
Shanghai AI Laboratory · 2023-07-06 · 7.3B parameters
InternLM 7B is the first-generation dense base model in the InternLM family, released as an open-weight checkpoint on July 6, 2023. Its official release says it was pretrained on trillions of high-quality tokens, but does not publish an exact cumulative token count. The checkpoint uses 32 decoder blocks, full multi-head attention, dynamic NTK-scaled RoPE with a 2,048-token configured maximum, and a SiLU-gated MLP.
Benchmark scores
| Benchmark | Score |
|---|---|
| ARC AI2 | 69.5 |
| BBH (BIG-Bench Hard) | 37.0 |
| BoolQ | 64.1 |
| Capability | 90.9 |
| Epoch Capabilities Index (ECI) | 102 |
| GSM8K | 31.2 |
| HellaSwag (Unspecified Scoring Protocol) | 70.6 |
| LAMBADA | 67.0 |
| MMLU | 51.0 |
| PIQA | 77.9 |