Gemini 1.5 Pro Experimental 0801
Google DeepMind · 2024-08-01
Gemini 1.5 Pro Experimental 0801 was Google's August 1, 2024 experimental update to Gemini 1.5 Pro, released through Google AI Studio and the Gemini API for early testing and feedback. It accepts text, image, audio, and video input, returns text, supports a 2,097,152-token input limit, and Google reported that it ranked first on the LMSYS leaderboard for both text and multimodal queries shortly after release.
Benchmark scores
| Benchmark | Score |
|---|---|
| BigCodeBench-Hard Complete | 29.7 |
| BigCodeBench-Hard Instruct | 25.0 |
| Creative Writing v2 | 80.3 |
| ZeroEval CRUX | 74.9 |
| ZeroEval GSM8K | 95.0 |
| ZeroEval MMLU-Redux | 85.5 |
| ZeroEval ZebraLogic | 25.2 |