Atlas

Models

← All models

Gemma 1.1 7B IT

Google DeepMind · 2024-04-05 · 8.5B parameters

Gemma 1.1 7B IT is Google's updated instruction-tuned 7B Gemma checkpoint. Google says it uses a novel RLHF method to improve quality, coding, factuality, instruction following, and multi-turn conversation quality, and fixes a multi-turn conversation bug in the original instruction-tuned release.

Benchmark scores

BenchmarkScore
Arena Text — Business, Management & Financial Ops (No Style Control)1063
Arena Text — Chinese (No Style Control)1060
Arena Text — Creative Writing (No Style Control)1060
Arena Text — English (No Style Control)1124
Arena Text — Entertainment, Sports & Media (No Style Control)1050
Arena Text — Exclude Ties (No Style Control)919
Arena Text — Exclude Ties (Style Controlled)1040
Arena Text — Expert (No Style Control)1039
Arena Text — French (No Style Control)1064
Arena Text — French (Style Controlled)1173
Arena Text — German (No Style Control)1054
Arena Text — Hard Prompts (No Style Control)1071
Arena Text — Hard Prompts English (No Style Control)1093
Arena Text — Instruction Following (No Style Control)1057
Arena Text — Japanese (No Style Control)971
Arena Text — Korean (No Style Control)988
Arena Text — Legal & Government (No Style Control)1094
Arena Text — Life, Physical & Social Science (No Style Control)1078
Arena Text — Longer Query (No Style Control)1056
Arena Text — Medicine & Healthcare (No Style Control)1072
Arena Text — Multi-turn (No Style Control)1040
Arena Text — Non-English (No Style Control)1051
Arena Text — Non-English (Style Controlled)1142
Arena Text — Occupational Mathematical (No Style Control)1098
Arena Text — Russian (No Style Control)1045
Arena Text — Software & IT Services (No Style Control)1096
Arena Text — Spanish (No Style Control)1048
Arena Text — Writing, Literature & Language (No Style Control)1073
BBH (Open LLM Leaderboard v2)39.4
Capability101.3
Loading Atlas data…