Name: Gemma 7B
Author: Google DeepMind

Gemma 7B

Google DeepMind · 2024-02-21 · 8.5B parameters

Gemma 7B is Google's ~8.5-billion-parameter open-weight model from the original Gemma family, released in February 2024. It is a text-only decoder-only model derived from the same research and technology as the Gemini models, designed for responsible on-device and API-based deployment.

Benchmark scores

Benchmark	Score
Adversarial NLI	48.7
BBH (BIG-Bench Hard)	55.1
BBH (Open LLM Leaderboard v2)	43.6
BoolQ	83.2
Capability	104.2
Epoch Capabilities Index (ECI)	111
GPQA (Open LLM Leaderboard v2)	28.7
GSM8K	46.4
IFEval (Open LLM Leaderboard v2)	26.6
MATH Level 5 (Open LLM Leaderboard v2)	7.4
MedMCQA (Open Medical-LLM)	49.3
MedQA 4-options (Open Medical-LLM)	48.7
MMLU Anatomy (Open Medical-LLM)	58.5
MMLU Clinical Knowledge (Open Medical-LLM)	69.4
MMLU College Biology (Open Medical-LLM)	77.1
MMLU College Medicine (Open Medical-LLM)	60.7
MMLU Medical Genetics (Open Medical-LLM)	70.0
MMLU Professional Medicine (Open Medical-LLM)	63.2
MMLU-Pro (Open LLM Leaderboard v2)	29.5
MuSR (Open LLM Leaderboard v2)	40.6
OpenBookQA	78.6
PIQA	81.2
PubMedQA (Open Medical-LLM)	75.6