Models

← All models

InternLM 20B

Shanghai AI Laboratory · 2023-09-20 · 20.1B parameters

InternLM 20B is the first-generation base checkpoint released by Shanghai AI Laboratory and collaborators on September 20, 2023. It is a dense 60-layer decoder-only text model pretrained on more than 2.3 trillion English, Chinese, and code tokens, using a SentencePiece byte-level BPE tokenizer and dynamic rotary position embeddings. The checkpoint declares 4,096 native positions, while the official model card says it supports 16k context through inference-time extrapolation without stating an exact maximum token count.

Benchmark scores