面向推測解碼的驗證感知訓練

摘要

推測解碼透過使用草稿模型生成候選詞元，再由目標模型在單次前向傳播中進行驗證，從而加速大型語言模型推論。驗證過程依序進行，並從第一次拒絕的位置起丟棄所有後續位置；然而現有的草稿訓練依賴對目標模型的詞元級模仿，並使用固定的逐位置加權，既未反映驗證的序貫性，也未反映拒絕後的丟棄行為。我們提出驗證感知訓練（Verification-Aware Training, VAT），這是一個插件式框架，在每個訓練步驟模擬驗證過程，並將產生的接受與拒絕模式轉化為監督信號。VAT 包含兩個組成部分：(i) 驗證頭（verification head），一個輕量且聯合訓練的二元分類器，用於監督草稿模型每個位置是否能通過序貫驗證；(ii) 驗證自適應加權（verification-adaptive weighting），以每個樣本第一次被拒絕的位置為界，在此之前保留完整權重，並重新將衰減的起點定錨於該位置，取代原先固定的加權機制。VAT 僅修改訓練目標，因此可疊加於現有方法之上，無需更改草稿模型架構、目標模型或推論流程。在 Qwen3-4B、Qwen3-8B 與 LLaMA-3.1-8B 上應用於 EAGLE-3 和 DFlash，VAT 將平均接受長度提升最高 11.4%，實際耗時加速最高 8.7%，並在數學、程式碼與聊天基準上均取得一致提升。程式碼將於 https://github.com/naver-ai/vat 公開。

English

Speculative decoding accelerates large language model inference by using a draft model to generate candidate tokens, which are verified by the target model in a single forward pass. Verification proceeds sequentially and discards every position from the first rejection onward, yet existing draft training relies on token-level imitation of the target with a fixed per-position weighting that reflects neither property. We introduce Verification-Aware Training (VAT), a plug-in framework that simulates verification at every training step and turns the resulting accept and reject patterns into supervision. VAT consists of two components: (i) a verification head, a lightweight jointly trained binary classifier that supervises the draft model on whether each position survives sequential verification; (ii) verification-adaptive weighting, which replaces the fixed weighting schedule by keeping full weight up to each sample's first rejection point and re-anchoring the decay to start there. VAT modifies only the training objective, so it can be layered on top of existing methods without changing the draft architecture, the target model, or the inference procedure. Applied to EAGLE-3 and DFlash on Qwen3-4B, Qwen3-8B, and LLaMA-3.1-8B, VAT improves average acceptance length by up to 11.4% and wall-clock speedup by up to 8.7%, with consistent gains across math, code, and chat benchmarks. Code will be available at https://github.com/naver-ai/vat