Cerebras-GPT 13B
Cerebras Systems · 2023-03-28 · 12.9B parameters
Cerebras-GPT 13B is the largest checkpoint in Cerebras Systems' open, compute-optimal GPT family, released on March 28, 2023 and trained on the Pile with a GPT-3-style decoder-only architecture. Its training exposure was reported as 257.1 billion tokens under the Chinchilla scaling recipe on the 16-CS-2 Andromeda AI supercomputer. Cerebras released the weights, training code, and checkpoints under the Apache 2.0 license.
Benchmark scores
| Benchmark | Score |
|---|---|
| Epoch Capabilities Index (ECI) | 82.4 |