Jamba 1.5 Large
AI21 Labs · 2024-08-22 · 398.6B parameters
Jamba 1.5 Large is AI21 Labs' instruction-tuned hybrid Transformer-Mamba mixture-of-experts model, released on August 22, 2024, with nine eight-layer blocks, a 1:7 attention-to-Mamba ratio, 16 experts, and top-2 routing. AI21 reports approximately 398B total and 94B active parameters and a 256K-token effective context window; the model supports function calling, structured JSON output, grounded generation, and nine languages.