Name: Gemini 1.0 Pro
Author: Google DeepMind

Gemini 1.0 Pro

Google DeepMind · 2023-12-13

Gemini 1.0 Pro (`gemini-pro`) was the first developer-focused Pro model in Google's Gemini 1.0 family, made available through the Gemini API on December 13, 2023. The text-in/text-out endpoint used a 32,768-token total sequence, with the API exposing 30,720 input tokens and up to 2,048 output tokens; Google documents SentencePiece tokenization and multi-query attention for Gemini 1.0.

Benchmark scores

Benchmark	Score
AI2D (OpenVLM)	72.9
AlpacaEval 2.0 (length-controlled win rate)	24.4
AlpacaEval 2.0 (raw win rate)	18.2
Arena Text — Business, Management & Financial Ops (No Style Control)	1083
Arena Text — Chinese (No Style Control)	1095
Arena Text — Creative Writing (No Style Control)	1104
Arena Text — English (No Style Control)	1154
Arena Text — Entertainment, Sports & Media (No Style Control)	1086
Arena Text — Exclude Ties (No Style Control)	978
Arena Text — Exclude Ties (Style Controlled)	1104
Arena Text — Hard Prompts (No Style Control)	1094
Arena Text — Hard Prompts English (No Style Control)	1113
Arena Text — Instruction Following (No Style Control)	1110
Arena Text — Legal & Government (No Style Control)	1108
Arena Text — Life, Physical & Social Science (No Style Control)	1119
Arena Text — Medicine & Healthcare (No Style Control)	1116
Arena Text — Multi-turn (No Style Control)	1139
Arena Text — Non-English (No Style Control)	1094
Arena Text — Non-English (Style Controlled)	1191
Arena Text — Occupational Mathematical (No Style Control)	1136
Arena Text — Software & IT Services (No Style Control)	1112
Arena Text — Writing, Literature & Language (No Style Control)	1127
Artificial Analysis Intelligence Index	3.1
BBH (BIG-Bench Hard)	75.0
Capability	109.3
Coding	685
Epoch Capabilities Index (ECI)	116
EQ-Bench v2	75.1
EQ-Bench v2 + MAGI-Hard Combined	61.0
GSM1k	79.8