Paper recorded by Signals 4 on 2026-10-06 in cs.AI. Abstract reproduced from arXiv; link to the original below.
Published 2026-10-06 on arXiv · recorded by Signals 4 on 2026-10-07
Category: cs.AI · 人工智能 · first seen 2026-10-07
Speculative decoding accelerates inference for a large language model (LLM), referred to as the \emph{target model}, by first using a smaller model, referred to as the \emph{draft model}, to generate candidate tokens and then verifying them with the target model for acceptance or rejection. Prior studies primarily focused on the efficiency-utility trade-off of speculative decoding, e.g., lossy spe