Paper recorded by Signals 4 on 2026-09-17 in cs.LG. Abstract reproduced from arXiv; link to the original below.
Published 2026-09-17 on arXiv · recorded by Signals 4 on 2026-09-18
Category: cs.LG · 机器学习 · first seen 2026-09-18
Adapting a pretrained autoregressive (AR) model is a cost-efficient route to a diffusion language model (DLM). While nearly all such adaptations start from a full-attention transformer, AR modeling has shifted toward hybrid architectures that interleave attention and RNN layers. This creates an obstacle for adaptation: unlike attention, RNNs are structurally causal and nontrivial to bidirectionali