Gemini 1.0 Ultra
Google DeepMind · 2024-02-15
Gemini 1.0 Ultra was Google's hosted text model for complex instruction following, coding, reasoning, and multilingual tasks. The gemini-ultra endpoint became generally available to allowlisted Vertex AI customers on February 15, 2024, with an 8,192-token maximum input and a 2,048-token maximum output.
Benchmark scores
| Benchmark | Score |
|---|---|
| Artificial Analysis Coding Index | 17.6 |
| Artificial Analysis Intelligence Index | 4.6 |
| BBH (BIG-Bench Hard) | 83.6 |
| Capability | 120.9 |
| HellaSwag (Unspecified Scoring Protocol) | 87.8 |
| HumanEval | 74.4 |
| MATH | 53.2 |
| MathVista (mini) | 53.0 |
| MGSM | 79.0 |