Judging by the Cover: Cleaning LLM Truthfulness Benchmarks to Avoid Surface-Level Feature Leakage
Paper recorded by Signals 4 on 2026-09-11 in cs.CL. Abstract reproduced from arXiv; link to the original below.
Published 2026-09-11 on arXiv · recorded by Signals 4 on 2026-09-14
Category: cs.CL · 自然语言处理 · first seen 2026-09-14
Abstract
Binary-choice truth benchmarks ask models to choose between a correct and an incorrect answer, but if the two answers differ systematically in surface-level features, models can exceed chance without performing the intended reasoning. We show that this failure mode is detectable and can be exploited by downstream classifiers. In TruthfulQA, a simple six-feature logistic classifier achieves substan
Read on arXiv →
Cite this page: Judging by the Cover: Cleaning LLM Truthfulness Benchmarks to Avoid Surface-Level Feature Leakage: the #56 most recent of 186 cs.CL papers we have recorded (as of 2026-09-11). Source: Signals 4 (Signals API) — https://data.jiangzhang.ca/signals4/t/papers/judging-by-the-cover-cleaning-llm-truthfulness-benchmarks-to-avoid-surface-level.html
Free to quote with attribution to “Signals 4 (Signals API)”. Machine-readable:
papers.json
Get 4 AI signals a day by email — free.
Get 4 AI signals a day by email — free