用於可組合非結構化知識編輯的混合策略自編輯

摘要

大型語言模型（LLM）在自然語言任務中展現出卓越的效能，然而它們是以靜態語料庫進行訓練，在快速變遷的世界中，其知識很快便會過時。這促使了知識編輯（KE）的發展，其旨在更新LLM中的特定知識，同時不影響其他無關的知識。近期研究從結構化的知識三元組轉向非結構化知識編輯（UKE），其中編輯內容為一段自由形式的文本，可能同時陳述多個事實。儘管如此，現有的編輯器雖能注入此類文本，卻未能善加利用：編輯後的模型雖能回憶該文本，卻既無法回答關於其中事實的原子性問題，也無法將這些事實組合成多跳推理。我們將此缺失的屬性稱為可組合性，並將其歸因於編輯器被動地依賴固定文本作為唯一學習來源。為此，我們將編輯重新詮釋為一種從同一模型的特權情境內狀態進行的主動式自蒸餾，此過程無需外部監督。我們進一步揭示，由於所注入知識的新穎性，編輯前模型自身的軌跡展開（rollout）鮮少涵蓋這些知識，從而限制了純粹同策略蒸餾的有效性。為填補此差距，我們提出HPSE，其建構一種混合式軌跡展開，在學生的自身軌跡覆蓋不足之處精準介入，將缺失的事實置入其中，同時在其他位置保持同策略。我們在理論上分析了HPSE相較於純粹同策略蒸餾的優勢，並在實證上確立了其在四種LLM主幹模型與兩種KE編輯器於各種情境下的即插即用改進。

English

Large language models (LLMs) achieve remarkable performance across natural language tasks, yet they are trained on static corpora and their knowledge quickly becomes outdated in a fast-changing world. This motivates knowledge editing (KE), which updates specific knowledge in an LLM without changing unrelated others. Recent works move from structured knowledge triples toward unstructured KE (UKE), where the edit is a free-form passage that may state multiple facts at once. Nonetheless, existing editors inject such a passage yet fail to use it: the edited model can recall the passage, but can neither answer atomic questions about its facts nor compose them into multi-hop reasoning. We attribute this missing property, which we term composability, to editors' passive reliance on the fixed passage as the sole learning source. In response, we cast editing as a proactive self-distillation from a privileged in-context state of the same model, which requires no external supervision. We further reveal that due to the novelty of the injected knowledge, the pre-edited model's own rollouts rarely cover it, which limits the effectiveness of pure on-policy distillation. To close this gap, we propose HPSE, which builds a hybrid rollout that steps in to place missing facts onto the student's own trajectory precisely where its coverage fails, while staying on-policy elsewhere. We theoretically analyze HPSE's advantage over pure on-policy distillation, and empirically establish its plug-and-play improvements across four LLM backbones and two KE editors under various scenarios.