Signals 4 · free daily AI digest

Holdout Best-of-N: Unbiased Evaluation and Its Cost

Paper recorded by Signals 4 on 2026-10-06 in cs.CL. Abstract reproduced from arXiv; link to the original below.

Published 2026-10-06 on arXiv · recorded by Signals 4 on 2026-10-07

Category: cs.CL · 自然语言处理 · first seen 2026-10-07

Abstract

Reusing the scores that select a Best-of-$N$ winner can overstate its expected reward. We study evaluation from a fixed matrix of $K$ independent scores per candidate for a policy that selects using $J$ fresh scores. A single estimator based only on this matrix is exactly unbiased for expected judge reward under every independent, stable collection of candidate-specific score laws if and only if $

Read on arXiv →

#309 most recent of 660 cs.CL papers we have recorded · ↑ newer: A Systematic Study of Semantic ID Spaces for Generative Information Re · ↓ older: Disentangling Paradigm, Identifier, and Decoding in Generative Retriev
Cite this page: Holdout Best-of-N: Unbiased Evaluation and Its Cost: the #309 most recent of 660 cs.CL papers we have recorded (as of 2026-10-06). Source: Signals 4 (Signals API) — https://data.jiangzhang.ca/signals4/t/papers/holdout-best-of-n-unbiased-evaluation-and-its-cost.html
Free to quote with attribution to “Signals 4 (Signals API)”. Machine-readable: papers.json
Related: More cs.CL papers · arXiv signals · All papers · Today in AI
Get 4 AI signals a day by email — free.
Subscribe free → See all plans →
Get 4 AI signals a day by email — free
All models · All repos · By company · Daily editions