Vals RSI — Harness Engineering
Agents · 2026-09-21
Reference-normalized judge-harness research score after 12 hours. Agents improve a fixed Gemini 3.5 Flash-Lite judge using 12 development apps; frozen harness evaluated three times on 24 held-out apps with 205 substeps. Agreement baseline 50%, reference 95%, theoretical best 99%.
Top models (higher is better)
| Model | Score |
|---|---|
| Claude Fable 5.1 | 16.4 |
| Grok 4.6 | 15.7 |
| GPT-5.6 Sol | 12.9 |
| Claude Fable 5 | 12.6 |
| GLM-5.3 | 12.0 |
| Opus 4.8 | 11.4 |
| Claude Opus 5 | 9.9 |
| Claude Opus 5.5 | 9.6 |
| Gemini 3.8 Flash | 9.6 |
| Muse Spark 1.3 | 9.6 |
| Kimi K3 | 6.0 |
| GPT-6 Astra | 5.7 |
| GPT-5.4 | 5.2 |
| Opus 4.7 | 3.0 |
| GPT-5.5 | 3.0 |
| GPT-5.2 | 1.9 |
| Kimi K2.5 | 0.9 |