Same Sentence, Different Construct: Identity-Swapped Rationales Reveal Measurement Non-Invariance in LLM Hate Speech Classifiers

Paper recorded by Signals 4 on 2026-11-02 in cs.CL. Abstract reproduced from arXiv; link to the original below.

Published 2026-11-02 on arXiv · recorded by Signals 4 on 2026-11-03

Category: cs.CL · 自然语言处理 · first seen 2026-11-03

Abstract

Large language models are increasingly used as measurement instruments for socially consequential constructs such as hate speech, and are typically validated by their aggregate agreement with human labels. A valid instrument must also apply the same construct to every group it measures. We test this requirement using 421 \hc{} templates that are instantiated identically for seven protected groups,

Read on arXiv →