Paper recorded by Signals 4 on 2026-11-02 in cs.CL. Abstract reproduced from arXiv; link to the original below.
Published 2026-11-02 on arXiv · recorded by Signals 4 on 2026-11-03
Category: cs.CL · 自然语言处理 · first seen 2026-11-03
Rewards for diversity, coverage and distribution matching make the reward of each sampled response depend on the rest of its group, and they are now routinely optimized with GRPO and RLOO. We prove that in this setting the group-relative baseline changes the objective. The expected update equals each response's own reward gradient plus $α$ times its effect on its groupmates' rewards, with $α=-1/(K