How Does mHC Use Its Residual Streams? Selective Routing and Near-Identity Mixing

Paper recorded by Signals 4 on 2026-09-04 in cs.AI. Abstract reproduced from arXiv; link to the original below.

Published 2026-09-04 on arXiv · recorded by Signals 4 on 2026-09-07

Category: cs.AI · 人工智能 · first seen 2026-09-07

Abstract

Hyper-Connections and their manifold-constrained variant mHC widen a residual pathway from one stream to n, yet how trained models use this capacity remains unclear: how broadly blocks read and write, how strongly the residual pathway mixes streams, and whether the streams carry distinct representations. We examine these properties in the four-stream residual pathway of DeepSeek-V4-Flash using eff

Read on arXiv →