Name: Gemini 1.5 Flash
Author: Google DeepMind

Gemini 1.5 Flash

Google DeepMind · 2024-05-10

Gemini 1.5 Flash is Google's lightweight, natively multimodal transformer-decoder model, designed for efficient TPU serving with lower latency and first released in preview on May 10, 2024. Google documented the unversioned `gemini-1.5-flash` ID as an alias that it redirected to version 002 in November 2024; the hosted model accepted text, images, audio, and video, returned text, and exposed a 1,048,576-token input limit.

Benchmark scores

Benchmark	Score
A-Fantasia	74.7
A-Fantasia Backwards Spelling	100.0
A-Fantasia Chess	58.0
A-Fantasia Cube Rotation	66.0
AI2D (OpenVLM)	78.5
AidanBench	1014
Artificial Analysis Intelligence Index	4.9
BBH (BIG-Bench Hard)	85.5
BoolQ	85.8
Capability	119.1
CorpFin v2	28.6
CorpFin v2 - Exact Pages	38.5
CorpFin v2 - Max Fitting Context	26.0
CorpFin v2 - Shared Max Context	21.4
Creative Writing v2	72.0
Epoch Capabilities Index (ECI)	122
ForecastBench Tournament Overall Brier Index	53.9
Global MMLU Lite	73.7
HallusionBench (OpenVLM)	48.5
HardGeoBench <=100mi	32.5
HardGeoBench <=10mi	16.3
HardGeoBench Avg Distance	1763
HardGeoBench Country Accuracy	48.8
HellaSwag (Unspecified Scoring Protocol)	86.5
HumanEval	74.3
Humanity's Last Exam (Text-Only)	4.2
LiveCodeBench	30.7
LiveCodeBench (Artificial Analysis)	19.6
LLM Deception Disinformation Effectiveness	0.6
LLM Deception Vulnerability Score	0.6