Linear Probes Transfer from Base to Distilled Language Models despite Rotation

Paper recorded by Signals 4 on 2026-10-26 in cs.CL. Abstract reproduced from arXiv; link to the original below.

Published 2026-10-26 on arXiv · recorded by Signals 4 on 2026-10-27

Category: cs.CL · 自然语言处理 · first seen 2026-10-27

Abstract

Linear probes, classifiers that read a language model's internal activations, are widely used to detect harmful requests. They are cheap to train and run, and an auditor who has trained one would like to reuse it when the model is replaced by a successor rather than retrain it for every release. We test how well probes can be reused. For four base models (Qwen2.5-Math-7B, Llama-3.1-8B, Qwen2.5-14B

Read on arXiv →