量化感知修復：恢復壓縮4位元大型語言模型的實用配方

摘要

為了廉價地提供大型語言模型服務，日益依賴於同時將模型進行結構壓縮至原有參數量的極小比例，並量化至4位元。這些步驟共同導致推理、數學、程式設計與長上下文行為的退化，因此在部署之前需要一個恢復或修復階段。預設的配方——量化感知訓練（QAT）——會將壓縮且量化後的模型重新擬合至硬標籤；在我們的流程中，該方法收斂緩慢，且在超過峰值後急劇崩潰。我們改採用量化感知修復（QAH）。由於結構壓縮後的模型從未以全精度獨立訓練，其bfloat16檢查點是原始模型的蒸餾恢復近似；QAH直接將4位元學生模型從原始未壓縮模型蒸餾而得。在GPT-OSS 120B到60B再到MXFP4的流程中，QAH學生模型在9項基準測試中的7項達到或超越其bfloat16來源模型的表現，同時權重記憶體約減少至四分之一，參數量僅為教師模型的一半，並以Hypernova-60B之名對外發布開放權重。相較於匹配的QAT基線，QAH約快7倍達到可比峰值，且在持續訓練下保持穩定，無需手動調整的早期停止機制。我們也報告部署方面的經驗教訓，包括分散式訓練後端之間存在顯著且可重現的品質差距。我們的目標是提供一套無需數週超參數搜尋即可部署的配方。

English

Serving large language models cheaply increasingly means shipping models that are both structurally compressed to a fraction of their parameters and quantized to 4 bits. Together these steps degrade reasoning, mathematics, coding, and long-context behavior enough to require a recovery, or healing, stage before deployment. The default recipe, quantization-aware training (QAT), re-fits the compressed, quantized model to hard labels; in our pipeline it converged slowly and collapsed past its peak. We adopted Quantization-Aware Healing (QAH) instead. Because a structurally compressed model is never independently trained at full precision, its bfloat16 checkpoint is a distillation-recovered approximation of the original; QAH distills the 4-bit student directly from the original, uncompressed model. On a GPT-OSS 120B to 60B to MXFP4 pipeline, the QAH student matches or beats its bfloat16 source on 7 of 9 benchmarks at roughly 4 times less weight memory and half the teacher's parameter count, and is released open-weight as Hypernova-60B. Against a matched QAT baseline it reaches a comparable peak about 7 times faster and stays stable under continued training, without hand-tuned early stopping. We also report deployment lessons, including a large, reproducible quality gap between distributed-training backends. Our aim is a recipe deployable without a multi-week hyper-parameter search.