PLE (Physical-Engineering's Last Exam)
Professional Work · 2026-10-21
85 long-horizon agentic tasks in professional physical engineering (27 electrical, 18 civil, 30 mechanical, 10 structural) from Google Cloud AI Research: 44 expert-authored tasks built with Mercor and scored as the weighted fraction of binary rubric items passed, judged by Claude Opus 5.5 from the deliverable file, and 41 co-authored tasks scored by frozen executable graders as 0.4 x constraints passed + 0.6 x hard-constraint gate x normalised objective. Agents work offline for up to six hours per run; a run without a gradable deliverable scores 0. The score is the mean task score over three runs, in percent. 29 tasks are public; the rest are held out for a leaderboard.
Top models (higher is better)
| Model | Score |
|---|---|
| Claude Opus 5.5 | 50.6 |
| GPT-6 Astra | 48.3 |
| Claude Sonnet 5.5 | 45.2 |
| Gemini 4 Argon | 43.2 |
| Claude Opus 5 | 42.6 |
| Claude Fable 5 | 34.9 |
| Grok 4.7 | 32.0 |
| Gemini 3.8 Flash | 28.2 |
| GLM-5.2 | 25.3 |
| Gemini 3.1 Pro Preview | 14.2 |
| Qwen3-Coder-480B-A35B-Instruct | 6.2 |
| Kimi K2 Thinking | 5.9 |
| MiniMax M2 | 5.3 |
| Llama 4 Maverick Instruct | 0.7 |