EmbodiedLab (T1 success, perception and planning)
Agents · 2026-10-26
EmbodiedLab's second evaluation level, T1: the same 30 experiments, task cards, budgets and gates as T0, with the client adding point-prompted SAM 3 segmentation, ground-truth depth back-projection of chosen pixels and masks, and Cartesian hand motion planned collision-free by cuRobo against the bench's true collision model. Grasping, placing and every other manipulation skill stay with the agent. The score is the T1 success rate, averaged over each experiment's runs and then over the experiments, in percent.
Top models (higher is better)
| Model | Score |
|---|---|
| Claude Opus 5.5 | 53.3 |
| GPT-6 Astra | 46.7 |
| GPT-6.1 Sol | 30.0 |
| Kimi K3 | 20.0 |
| DeepSeek V4.1 Flash 1017 | 13.3 |
| GLM-5.3 Flash | 3.3 |