JustFit: 200K-Token LLM Serving on a 24 GiB Laptop with Just-in-Time State Management

Paper recorded by Signals 4 on 2026-09-15 in cs.AI. Abstract reproduced from arXiv; link to the original below.

Published 2026-09-15 on arXiv · recorded by Signals 4 on 2026-09-16

Category: cs.AI · 人工智能 · first seen 2026-09-16

Abstract

Capable open-weight models make local coding and reasoning attractive, but their context and execution state strain laptop memory. We present JustFit, an MLX-based inference runtime that combines KVExec for compressed KV execution, PhaseSwap for component residency, and StateTrans for state-preserving serving transitions. These mechanisms fuse reconstruction and coordinate just-in-time materializa

Read on arXiv →