Paper recorded by Signals 4 on 2026-09-17 in cs.LG. Abstract reproduced from arXiv; link to the original below.
Published 2026-09-17 on arXiv · recorded by Signals 4 on 2026-09-18
Category: cs.LG · 机器学习 · first seen 2026-09-18
Reinforcement learning (RL) of large language models is notoriously sensitive to small differences between training and inference engines, often referred to as the training-inference mismatch (TIM). However, completely eliminating TIM is impractical, as it would come at a major cost to rollout efficiency. In this paper, we show that the instability of RL under TIM is primarily caused by drift: a p