Models

← All models

Chinchilla

Google DeepMind · 2022-03-29

Chinchilla is DeepMind's dense autoregressive language model, introduced in March 2022 and reported as 70B parameters trained on 1.4 trillion tokens with the same compute budget as Gopher. It established influential compute-optimal scaling results: increasing training data while reducing model size let it outperform the much larger Gopher across a broad evaluation suite. Both the parameter and token counts are published at rounded precision, and although training used 2,048-token sequences, no exact maximum supported context was stated.

Benchmark scores