InternLM 7B
Shanghai AI Laboratory · 2023-07-06 · 7.3B parameters
InternLM 7B is the first-generation dense base model in the InternLM family, released as an open-weight checkpoint on July 6, 2023. Its official release says it was pretrained on trillions of high-quality tokens, but does not publish an exact cumulative token count. The checkpoint uses 32 decoder blocks, full multi-head attention, dynamic NTK-scaled RoPE with a 2,048-token configured maximum, and a SiLU-gated MLP.
Benchmark scores
| Benchmark | Score |
|---|---|
| Epoch Capabilities Index (ECI) | 102 |