RedPajama INCITE 7B Base
Together AI · 2023-06-06 · 6.9B parameters
RedPajama-INCITE-7B-Base is Together AI's 6.9-billion-parameter fully open pretrained language model, released in June 2023 as part of the collaborative RedPajama project. It was trained on 1 trillion tokens from the RedPajama dataset on 3,072 V100 GPUs provided through the DOE INCITE program, with development contributions from Together, MILA, LAION, EleutherAI, and Stanford CRFM. The model is released under Apache 2.0 and served as the base for the accompanying RedPajama-INCITE instruct and chat variants.