为何门控DeltaNet能经受4比特量化考验：混合27B大语言模型中递归部分的NVFP4 W4A4量化方案

摘要

混合大语言模型（hybrid LLM）将softmax注意力与线性注意力层（如门控DeltaNet，GDN）搭配使用，后者以固定大小的循环状态概括上下文。社区早期针对Qwen3.8-27B（含48个GDN层与16个注意力层）的4比特量化方案将GDN模块——尤其是其衰减（decay）门与写入强度（write-strength）门——保留为8比特或16比特精度，其直觉依据是循环结构中的误差会在长上下文上累积。我们通过构建Minima来检验这一直觉：对所有496个线性层（含GDN）一律采用NVFP4 W4A4量化。在4K/32K困惑度、MMLU-Pro、GSM8K、AIME'25、GPQA-Diamond、LiveCodeBench以及最长至64K的RULER检索任务中，Minima在随机种子波动范围内与BF16相当（5任务平均差值为−0.52），同时是我们所比较方案中体积最小（17.5 GiB）且预填充最快（+14%–19%）的配方；其在32K长度下的困惑度差距还随位置推进而收窄。一项由四部分构成的机制研究揭示了原因：（i）NVFP4的16元素块缩放将残差流中的极端离群值局限在局部，使不同角色层的激活误差趋于均衡；（ii）被认为最脆弱的门投影实际上最不敏感——softplus/指数与sigmoid参数化将约11%的GEMM误差压缩为约2%的输出误差；（iii）因为每次写入都会沿当前键方向覆写状态，delta规则循环能将注入噪声在32K个token上维持在平坦的平台期，并在数百步内遗忘状态脉冲；（iv）逐token量化成本随上下文变长而稀释，而非累积放大。我们还修复了一种全局缩放失配问题——当逐模块校准的NVFP4检查点由将这些模块融合为单个GEMM的内核服务时便会出现该失配——并证明了校准后的FP8 KV缓存缩放不带来性能开销。最终结果是一套实用配方——量化一切，并随附KV缩放——以及对“混合LLM的循环半边为何是更易于量化的半边”的机制性解释。检查点：https://huggingface.co/minima-ai/mnma_qwen3.8_27b_nvfp4

English

Hybrid LLMs pair softmax attention with linear-attention layers such as Gated DeltaNet (GDN), whose recurrent state summarizes the context in fixed size. Early community 4-bit quantizations of Qwen3.8-27B (48 GDN layers, 16 attention layers) left the GDN block in 8- or 16-bit precision -- especially its decay and write-strength gates -- on the intuition that errors in a recurrence accumulate over long contexts. We test that intuition by building Minima: NVFP4 W4A4 on all 496 linear layers, GDN included. Across perplexity at 4K/32K, MMLU-Pro, GSM8K, AIME'25, GPQA-Diamond, LiveCodeBench, and RULER retrieval to 64K, Minima matches BF16 within seed noise (5-task average -0.52) while being the smallest (17.5 GiB) and fastest-prefill (+14-19%) recipe we compare, and its 32K perplexity gap shrinks with position. A four-part mechanism study explains why: (i) NVFP4's 16-element block scaling localizes the residual stream's extreme outliers, equalizing activation error across layer roles; (ii) the supposedly fragile gate projections are the least sensitive -- softplus/exponential and sigmoid parameterizations compress ~11% GEMM error to ~2% output error; (iii) the delta-rule recurrence holds injected noise at a flat plateau over 32K tokens and forgets a state impulse within hundreds of steps, because each write overwrites the state along the current key direction; (iv) the per-token quantization cost washes out with context instead of compounding. We also repair a global-scale mismatch that arises when per-module-calibrated NVFP4 checkpoints are served by kernels that fuse those modules into one GEMM, and show calibrated FP8 KV-cache scales are performance-free. The result: a practical recipe -- quantize everything, ship KV scales -- and a mechanistic account of why the recurrent half of a hybrid LLM is the easy half to quantize. Checkpoint: https://huggingface.co/minima-ai/mnma_qwen3.8_27b_nvfp4