Prompts and probabilities in the linguistic assessment of language models: Evidence from semantic focus-sensitivity

Paper recorded by Signals 4 on 2026-10-29 in cs.CL. Abstract reproduced from arXiv; link to the original below.

Published 2026-10-29 on arXiv · recorded by Signals 4 on 2026-10-30

Category: cs.CL · 自然语言处理 · first seen 2026-10-30

Abstract

As language models (LMs) have improved, an increasingly common means of evaluating their linguistic competence is through explicit prompting (e.g., "Which sentence is more acceptable?"). Prior work indicates that such prompting underperforms compared to standard token-probability-based methods in smaller models, but improves in larger models. We test this in a new domain: semantic focus-sensitivit

Read on arXiv →