Atlas

Models

← All models

Mistral NeMo Instruct 2407

Mistral AI · 2024-07-18 · 12.2B parameters

Mistral NeMo Instruct 2407 is the instruction-fine-tuned checkpoint of the dense Mistral NeMo model, jointly trained by Mistral AI and NVIDIA and released on July 18, 2024. It is a text-only model with a 131,072-token context window and function-calling training, and uses the Tekken tokenizer with a 131,072-token vocabulary. Its downloadable weights are licensed under Apache-2.0; Mistral exposed the checkpoint as `open-mistral-nemo-2407` on la Plateforme.

Benchmark scores

BenchmarkScore
BALROG17.6
BBH (Open LLM Leaderboard v2)50.4
BFCL v4 Overall Accuracy27.6
BuseyBench SVG0.8
Capability111.9
Creative Writing v277.5
Creative Writing v3 Elo961
Creative Writing v3 Rubric Score9.5
CRED v0.928.0
CRED v0.92 - Data integrity0.0
CRED v0.92 - Execution65.0
CRED v0.92 - Paper–code1.1
CRED v0.92 - Prose–table4.0
Epoch Capabilities Index (ECI)118
EQ-Bench v277.1
EQ-Bench v2 + MAGI-Hard Combined60.4
FormationEval78.8
GPQA (Open LLM Leaderboard v2)29.0
IFEval (Open LLM Leaderboard v2)63.8
LisanBench Difficulty-Weighted11.5
LisanBench Path Length48.7
Longform Creative Writing28.7
MAGI-Hard43.6
MATH Level 5 (Open LLM Leaderboard v2)12.7
MMLU-Pro (Open LLM Leaderboard v2)35.2
MuSR (Open LLM Leaderboard v2)39.0
ObviousBench38.9
Slop Score54.4
SnakeBench Average Apples0.4
SnakeBench Best Apples3.0
Loading Atlas data…