← All benchmarks

HLAB-Vals-1.1

Professional Work · 2026-10-19

HLAB-Vals-1.1 is Vals' October 2026 rerun of Harvey's Legal Agent Benchmark on the 120-task private held-out set. Harvey fixed the holdout (17 unsolvable criteria removed, one mismatched source document replaced); the harness was re-synced to Harvey's upstream (explicit finish tool, reasoning-first judge schema, judge-error retries, Word comments visible to judges, pandoc 3.11) and Vals added context compaction, a finish nudge, a tool-result size cap, image reads and workspace-path expansion. Agents get six file tools and docx, pptx and xlsx skills with internet disabled, and every model was rerun from scratch. A task passes only if all its criteria pass; the score is the mean of two LLM judges' task pass rates (GPT 5.5 at medium reasoning and Claude Sonnet 4.6), giving each task 0, 0.5 or 1 credit. The 120 tasks are not binary outcomes or 240 independent judge observations. Not comparable with the v1 leaderboard (hlab).

Top models (higher is better)