GPT-2 (1.5B)
OpenAI · 2019-11-05 · 1.6B parameters
GPT-2 (1.5B) is the largest variant of OpenAI's second-generation language model, released in November 2019 with 1.5 billion parameters. It is a decoder-only transformer trained on a large web-scraped text corpus (WebText) with a next-token prediction objective, notable at the time for its surprisingly fluent long-form text generation. GPT-2 was staged in its release due to misuse concerns and remains an important research baseline for language modeling.
Benchmark scores
| Benchmark | Score |
|---|---|
| ARC AI2 | 25.0 |
| BoolQ | 61.8 |
| Capability | 63.8 |
| HellaSwag (Unspecified Scoring Protocol) | 40.0 |
| METR Time Horizons | 0.0 |
| OpenBookQA | 22.4 |
| PIQA | 70.5 |
| WinoGrande | 58.3 |