Robo-bench
Agents · 2026-10-13
Vals Robo-bench v1 asks a general-purpose coding agent to control a simulated 7-DoF Franka Panda arm and a 29-DoF Unitree G1 humanoid in real-time MuJoCo from motor commands, robot state, datasheets and camera images alone, with no planner, grasping or balance controller. Each of 24 tasks (13 arm, 11 humanoid; 21 private) is one three-minute wall-clock episode on one seed, scored by ordered milestones worth 100; the score is the mean over all 24 tasks. This is milestone progress, not binary task accuracy.
Top models (higher is better)
| Model | Score |
|---|---|
| GPT-6.1 Sol | 28.3 |
| GPT-6 Astra | 23.3 |
| Claude Opus 5.5 | 14.6 |
| Muse Wonder Full | 10.4 |
| Claude Fable 5.1 | 5.8 |
| Gemini 4 Argon | 0.8 |
| Grok 4.7 | 0.2 |