Falcon 40B
Technology Innovation Institute · 2023-05-25 · 41.8B parameters
Falcon 40B is TII's 40B causal decoder-only base language model trained on 1 trillion tokens of RefinedWeb and curated corpora, released in 2023 under the Apache 2.0 license. It shares the same decoder architecture as the rest of the Falcon series — featuring multi-query attention and FlashAttention — and was one of the largest permissively licensed open-weight models at the time of its release.