Gemini 1.5 Pro Experimental 0827
Google DeepMind · 2024-08-27
Gemini 1.5 Pro Experimental 0827 was Google's August 27, 2024 experimental Gemini API and Google AI Studio snapshot, published as a quality-improved Gemini 1.5 Pro version after exp-0801. Google's contemporaneous API documentation grouped it under Gemini 1.5 Pro and specified audio, image, video, and text input, text output, and a 2,097,152-token input limit; it did not expose a distinct thinking mode.
Benchmark scores
| Benchmark | Score |
|---|---|
| Agentic Tool Use (Chat) | 51.3 |
| Agentic Tool Use (Enterprise) | 60.3 |
| Arabic | 1147 |
| BigCodeBench-Hard Complete | 31.8 |
| BigCodeBench-Hard Instruct | 27.0 |
| Capability | 129.6 |
| Chinese | 1120 |
| Coding | 1007 |
| GSM1k | 94.7 |
| Instruction Following | 84.2 |
| Japanese | 1093 |
| Korean | 60.3 |
| Spanish | 1067 |
| ZeroEval | 66.1 |
| ZeroEval CRUX | 79.6 |
| ZeroEval MATH Level 5 | 68.1 |
| ZeroEval MMLU-Redux | 86.1 |
| ZeroEval ZebraLogic | 30.5 |