SciCode
Science · 2024-07-12
SciCode is a scientist-curated benchmark with 80 research coding problems decomposed into 338 subproblems; the standard test split used by the reported leaderboard has 288 subproblems from 65 main problems. It measures whether models can generate executable scientific code from research-style specifications.
Top models (higher is better)
| Model | Score |
|---|---|
| Claude Fable 5 | 60.2 |
| Gemini 3.1 Pro Preview | 58.9 |
| Kimi K3 | 58.7 |
| Muse Spark 1.1 | 58.2 |
| GPT-5.6 Sol | 56.9 |
| GPT-5.4 | 56.6 |
| GPT-5.5 | 56.1 |
| Claude Opus 5 | 55.7 |
| Opus 4.7 | 54.5 |
| Grok 4.5 | 54.1 |
| GPT-5.6 Terra | 53.9 |
| Sonnet 5 | 53.6 |
| Opus 4.8 | 53.5 |
| Kimi K2.6 | 53.5 |
| Gemini 3.5 Flash | 53.1 |
| Gemini 3.6 Flash | 52.7 |
| GPT-5.6 Luna | 52.5 |
| Muse Spark | 51.5 |
| GLM-5.2 | 50.5 |
| Grok Build 0.1 | 50.2 |
| MiMo-V2.5-Pro | 50.2 |
| DeepSeek-V4-Pro | 50.0 |
| GPT-5.4 Mini | 49.9 |
| Qwen3.7-Max | 48.8 |
| GPT-5.5 Instant (2026-06-25 hosted snapshot) | 48.6 |
| Kimi K2.7 Code | 47.5 |
| Grok 4.3 | 47.3 |
| MiniMax M2.7 | 47.0 |
| GPT-5.4 Nano | 46.9 |
| Sonnet 4.6 | 46.8 |
| Qwen3.7-Plus | 45.5 |
| MiniMax M3 | 45.4 |
| DeepSeek-V4-Flash | 44.9 |
| GLM-5.1 | 43.8 |
| Gemma 4 31B IT | 43.4 |
| Haiku 4.5 | 43.3 |
| GPT-5.1 | 43.3 |
| MiMo-V2.5 | 43.1 |
| Gemini 2.5 Pro | 42.8 |
| Nova 2 Pro (Preview) | 42.7 |