Llama 3.1 8B
Meta · 2024-07-23 · 8B parameters
Llama 3.1 8B is Meta's pretrained 8B-class base language model from the July 2024 Llama 3.1 release. It is a dense, multilingual, decoder-only Transformer with a checkpoint-configured 131,072-token context window, grouped-query attention, and a tiktoken-based tokenizer; Meta reports about 15 trillion pretraining tokens and a December 2023 data cutoff for the family. It is intended as a foundation for fine-tuning and downstream natural-language generation rather than as the instruction-tuned chat variant.