Paper recorded by Signals 4 on 2026-09-11 in cs.LG. Abstract reproduced from arXiv; link to the original below.
Published 2026-09-11 on arXiv · recorded by Signals 4 on 2026-09-14
Category: cs.LG · 机器学习 · first seen 2026-09-14
A single SM utilization percentage can make an LLM inference workload look compute-saturated while hiding how much useful work is being done. The problem is not that the counter is wrong, but that it collapses several different mechanisms into one number. This is most severe during decode, where each request contributes only one new token and dense projection GEMMs become small-row matrix multipli