GTFormer: A Syntax-Free Graph Network Alternative to Transformers for 8192$\times$ Length Extrapolation in Linear Time

Paper recorded by Signals 4 on 2026-10-27 in cs.LG. Abstract reproduced from arXiv; link to the original below.

Published 2026-10-27 on arXiv · recorded by Signals 4 on 2026-10-28

Category: cs.LG · 机器学习 · first seen 2026-10-28

Abstract

Self-attention mechanisms have established the Transformer as the dominant backbone across natural language processing, source code intelligence, and automated theorem proving. However, all-to-all pairwise token comparison enforces a quadratic $\mathcal{O}(N^2)$ compute and memory barrier alongside severe softmax attention dilution as sequence length $N$ scales beyond the training horizon. Inspire

Read on arXiv →