Paper recorded by Signals 4 on 2026-10-21 in cs.CL. Abstract reproduced from arXiv; link to the original below.
Published 2026-10-21 on arXiv · recorded by Signals 4 on 2026-10-22
Category: cs.CL · 自然语言处理 · first seen 2026-10-22
A benchmark is not merely a collection of test questions; it naturally induces an empirical measure over difficulty. In this work, we develop this measure-based view of benchmark evaluation. This perspective allows us to identify and formalize two central failure modes: measure mismatch, where the benchmark difficulty distribution differs from the intended difficulty distribution, and coverage gap