Paper recorded by Signals 4 on 2026-10-30 in cs.CL. Abstract reproduced from arXiv; link to the original below.
Published 2026-10-30 on arXiv · recorded by Signals 4 on 2026-11-02
Category: cs.CL · 自然语言处理 · first seen 2026-11-02
When a language model is fine-tuned to follow instructions, or further pretrained on a new subject, it often gets worse at predicting standard web text. This is known as "forgetting." We ask whether this loss can be undone by simple, fixed adjustments to the model's final token predictions. Across 14 pairs of base and derived models, from 1B to 7B parameters, we fitted a fixed bonus or penalty for