量化感知修复：恢复压缩4位大语言模型的实用方案

摘要

以低成本服务大型语言模型日益意味着发布既结构压缩到其参数的一小部分、又量化到4比特的模型。这些步骤共同削弱了推理、数学、编码和长上下文行为，以至于需要在部署前进行恢复或愈合阶段。默认的量化感知训练（QAT）方法将压缩并量化后的模型重新拟合到硬标签；在我们的流程中，它收敛缓慢，并在越过峰值后便崩溃。我们转而采用量化感知愈合（QAH）。由于结构压缩的模型从未在全精度下独立训练，其bfloat16检查点是对原始模型的蒸馏恢复近似；因此QAH直接从未压缩的原始模型中蒸馏4比特学生模型。在GPT-OSS 120B到60B再到MXFP4的流程中，QAH学生模型在9个基准测试中的7个上达到或超过其bfloat16来源，权重内存约为其四分之一，参数量仅为教师模型的一半，并以Hypernova-60B的名义发布开放权重。与匹配的QAT基线相比，它以约7倍的速度达到相当峰值，且在持续训练下保持稳定，无需手动调优的早停。我们还报告了部署中的经验教训，包括分布式训练后端之间一个巨大且可复现的质量差距。我们的目标是提供一种无需数周超参数搜索即可部署的方案。

English

Serving large language models cheaply increasingly means shipping models that are both structurally compressed to a fraction of their parameters and quantized to 4 bits. Together these steps degrade reasoning, mathematics, coding, and long-context behavior enough to require a recovery, or healing, stage before deployment. The default recipe, quantization-aware training (QAT), re-fits the compressed, quantized model to hard labels; in our pipeline it converged slowly and collapsed past its peak. We adopted Quantization-Aware Healing (QAH) instead. Because a structurally compressed model is never independently trained at full precision, its bfloat16 checkpoint is a distillation-recovered approximation of the original; QAH distills the 4-bit student directly from the original, uncompressed model. On a GPT-OSS 120B to 60B to MXFP4 pipeline, the QAH student matches or beats its bfloat16 source on 7 of 9 benchmarks at roughly 4 times less weight memory and half the teacher's parameter count, and is released open-weight as Hypernova-60B. Against a matched QAT baseline it reaches a comparable peak about 7 times faster and stays stable under continued training, without hand-tuned early stopping. We also report deployment lessons, including a large, reproducible quality gap between distributed-training backends. Our aim is a recipe deployable without a multi-week hyper-parameter search.