← All benchmarks

ClaimBench

Professional Work · 2026-10-20

Vals ClaimBench v1 asks whether a 10-K excerpt supports a claim, contradicts it or gives insufficient evidence, over 1,960 claims in five construction categories (392 supported, 785 contradicted, 783 insufficient evidence; always answering the majority label scores about 40%). Gold labels come from the construction rule; excerpts range from a paragraph to about 16k tokens. The score is single-pass label accuracy over all claims. Language models make a label call and then a per-label probability call per claim. Diagnostic: the page mainly ranks single-call decision classifiers that Atlas excludes, and four of its five language models score 100%.

Top models (higher is better)