← All benchmarks

iPrefBench Interaction Recall

Agents · 2026-10-26

iPrefBench's interaction recall: the share of all ask-only requirements (details held only by the simulated user, in 76 of the 130 tasks) that the agent elicits through ask_user, pooled across tasks, in percent. A Claude Sonnet 5 judge splits the agent's messages into individual requests and matches each to a requirement. It measures whether the agent asks enough, not task outcome, so it is a diagnostic; the total number of ask-only requirements is not printed.

Top models (higher is better)