TokenGate: Adapting LLMs for Fast and Memory-Efficient Prefill with Learned Token Pruning

Paper recorded by Signals 4 on 2026-10-19 in cs.AI. Abstract reproduced from arXiv; link to the original below.

Published 2026-10-19 on arXiv · recorded by Signals 4 on 2026-10-20

Category: cs.AI · 人工智能 · first seen 2026-10-20

Abstract

Expert human readers process large inputs (e.g., code repositories and long documents) by skimming to closely read only what matters. On the other hand, language models process all input tokens through each model layer, resulting in significant processing costs for long-context inputs. We introduce TokenGate, a small, drop-in component that extends existing trained models to enable end-to-end lear

Read on arXiv →