Models

← All models

T5 11B

Google DeepMind · 2019-10-23 · 11.3B parameters

T5-11B is the largest released checkpoint in Google's original Text-to-Text Transfer Transformer family, introduced with the 2019 T5 study. It is a 24-layer-per-stack encoder-decoder Transformer pretrained for one million steps on a multi-task mixture whose unsupervised component applied span corruption to C4, using a shared SentencePiece vocabulary and 512-token input and target lengths. The original Google checkpoint contains 11,307,321,344 model parameters.

Benchmark scores