Paper recorded by Signals 4 on 2026-10-29 in cs.AI. Abstract reproduced from arXiv; link to the original below.
Published 2026-10-29 on arXiv · recorded by Signals 4 on 2026-10-30
Category: cs.AI · 人工智能 · first seen 2026-10-30
Predicting how many tokens a large language model (LLM) will decode, using only prefill-phase hidden states, is a key primitive for inference scheduling. We show that current benchmarks conflate predictor quality with task hardness: when the target model, dataset, or sampling configuration changes, reported accuracy collapses even though the predictor is unchanged. We trace this instability to an