Harvey LAB held-out set (Fireworks run)
Professional Work
Fireworks' run of Harvey's Legal Agent Benchmark on the 120-task private held-out set, published on the Specialized Intelligence Index. It uses Harvey's upstream generation harness (file tools with docx, pptx and xlsx skills) and the dual-judge profile of GPT 5.5 at medium reasoning and Claude Sonnet 4.6, with three runs per task. The all-pass headline gives a task 1 when both judges pass every criterion, 0.5 when one does and 0 otherwise. Same task set and judges as Vals' HLAB-Vals-1.1, but without Vals' harness additions, and Fireworks does not say whether it used Harvey's October holdout fix, so it is kept as a separate component.
Top models (higher is better)
| Model | Score |
|---|---|
| Grok 4.6 | 22.1 |
| GLM-5.3 | 14.7 |
| Kimi K3 | 13.1 |
| Ember-1 | 11.1 |
| Claude Opus 5 | 7.4 |
| Claude Opus 5.5 | 6.3 |
| DeepSeek V4.1 Flash | 5.7 |
| GPT-6 Astra | 5.1 |
| DeepSeek V4 Pro 0813 | 3.6 |
| GPT-6 Luna | 1.4 |
| GPT-6 Sol | 0.7 |
| GPT-5.6 Sol | 0.6 |