Two Clocks: Why Larger Rollout Batches Learn Slower but Fail on Schedule in Reinforcement Learning with Verifiable Rewards
Paper recorded by Signals 4 on 2026-10-13 in cs.AI. Abstract reproduced from arXiv; link to the original below.
Published 2026-10-13 on arXiv · recorded by Signals 4 on 2026-10-14
Category: cs.AI · 人工智能 · first seen 2026-10-14
Abstract
Reinforcement learning with verifiable rewards (RLVR) relies on costly model-generated responses, motivating larger rollout batches and square-root learning-rate scaling. Reported to approximately preserve progress per sample with one optimizer step per rollout, this rule is less understood in practical multi-step training. We develop a two-clock framework that separates the samples needed to make
Read on arXiv →
Cite this page: Two Clocks: Why Larger Rollout Batches Learn Slower but Fail on Schedule in Reinforcement Learning with Verifiable Rewards: the #327 most recent of 960 cs.AI papers we have recorded (as of 2026-10-13). Source: Signals 4 (Signals API) — https://data.jiangzhang.ca/signals4/t/papers/two-clocks-why-larger-rollout-batches-learn-slower-but-fail-on-schedule-in-reinf.html
Free to quote with attribution to “Signals 4 (Signals API)”. Machine-readable:
papers.json
Get 4 AI signals a day by email — free.
Get 4 AI signals a day by email — free