Mixtral 8x7B Instruct
Mistral AI · 2023-12-11 · 46.7B parameters
Mixtral 8x7B Instruct v0.1 is Mistral AI's instruction-following sparse mixture-of-experts checkpoint, released alongside the base model on December 11, 2023 and tuned with supervised fine-tuning followed by Direct Preference Optimization. Each of its 32 layers contains eight SwiGLU feed-forward experts and routes every token to two; Mistral reports rounded sizes of 46.7B total and 12.9B active parameters per token, while the released checkpoint contains exactly 46,702,792,704 parameters. It supports a 32,768-token context and English, French, Italian, German, and Spanish; its weights are under Apache-2.0, and Mistral retired the model on March 30, 2025.