Old Ideas, Novel Problems: The Instability of LLM-Based Novelty Evaluation

Paper recorded by Signals 4 on 2026-10-01 in cs.CL. Abstract reproduced from arXiv; link to the original below.

Published 2026-10-01 on arXiv · recorded by Signals 4 on 2026-10-02

Category: cs.CL · 自然语言处理 · first seen 2026-10-02

Abstract

Automated ideation systems are often evaluated on the novelty of the ideas they produce, and that judgment is increasingly delegated to large language models. Such judges are typically built ad hoc and validated, if at all, on human-authored papers rather than on the generated ideas they are meant to score. So, how do novelty judges perform? Not well. We present a systematic controlled study of

Read on arXiv →