Inoculation Midtraining with Learned Neologisms

Paper recorded by Signals 4 on 2026-09-14 in cs.CL. Abstract reproduced from arXiv; link to the original below.

Published 2026-09-14 on arXiv · recorded by Signals 4 on 2026-09-15

Category: cs.CL · 自然语言处理 · first seen 2026-09-15

Abstract

Large language models (LLMs) often learn both desirable and undesirable properties during post-training. We study whether midtraining, an earlier training stage, can shape which of these properties later generalise. We introduce Inoculation Midtraining, a technique that teaches a base model that unsafe behaviour belongs to a designated <quarantine_token> context, as indicated by the <quarantine_to

Read on arXiv →