Paper recorded by Signals 4 on 2026-09-28 in cs.AI. Abstract reproduced from arXiv; link to the original below.
Published 2026-09-28 on arXiv · recorded by Signals 4 on 2026-09-29
Category: cs.AI · 人工智能 · first seen 2026-09-29
Linear Vision Transformers (ViTs) are designed to replace the attention in Softmax ViTs with the linear-complexity attention operator for more efficient token routing, but they require from-scratch pre-training and typically underperform the original Softmax version. How to initialize linear ViTs both efficiently and effectively still remains unclear. In this work, we explicitly ask: given that mo