Paper recorded by Signals 4 on 2026-10-15 in cs.LG. Abstract reproduced from arXiv; link to the original below.
Published 2026-10-15 on arXiv · recorded by Signals 4 on 2026-10-16
Category: cs.LG · 机器学习 · first seen 2026-10-16
Reinforcement learning has become a widely used approach to post-training large language models. In a typical implementation, learning is off-policy: The policy generating the responses differs from the policy being trained because of training-inference mismatch or policy staleness in asynchronous training. This discrepancy is commonly addressed by importance sampling, whose ratio is usually regul