面向可组合非结构化知识编辑的混合策略自编辑方法

摘要

大语言模型（LLM）在自然语言任务中展现出卓越的性能，然而它们基于静态语料库进行训练，其知识在快速变化的世界中很快就会过时。这推动了知识编辑（KE）的发展，该技术在不改变无关知识的前提下更新LLM中的特定知识。近期研究从结构化知识三元组转向非结构化知识编辑（UKE），其中编辑内容是一段可以同时陈述多个事实的自由文本段落。然而，现有编辑器注入了这样的段落却未能加以利用：编辑后的模型能够回忆该段落，但既无法回答关于其事实的原子性问题，也无法将这些事实组合成多跳推理。我们将这种缺失的属性——称之为可组合性——归因于编辑器被动地依赖固定段落作为唯一学习来源。为此，我们将编辑转化为从同一模型的特权上下文状态进行的主动性自蒸馏，该方法无需外部监督。我们进一步揭示，由于注入知识的新颖性，编辑前模型自身的采样轨迹很少覆盖这些知识，这限制了纯在策略蒸馏的有效性。为弥补这一差距，我们提出了HPSE，它构建了一种混合轨迹，在学生的自身轨迹覆盖失败的位置精确介入以补充缺失的事实，同时在其他位置保持在策略内。我们从理论上分析了HPSE相较于纯在策略蒸馏的优势，并通过实验在四种LLM骨干模型和两种KE编辑器下、多种场景中确立了其即插即用的改进效果。

English

Large language models (LLMs) achieve remarkable performance across natural language tasks, yet they are trained on static corpora and their knowledge quickly becomes outdated in a fast-changing world. This motivates knowledge editing (KE), which updates specific knowledge in an LLM without changing unrelated others. Recent works move from structured knowledge triples toward unstructured KE (UKE), where the edit is a free-form passage that may state multiple facts at once. Nonetheless, existing editors inject such a passage yet fail to use it: the edited model can recall the passage, but can neither answer atomic questions about its facts nor compose them into multi-hop reasoning. We attribute this missing property, which we term composability, to editors' passive reliance on the fixed passage as the sole learning source. In response, we cast editing as a proactive self-distillation from a privileged in-context state of the same model, which requires no external supervision. We further reveal that due to the novelty of the injected knowledge, the pre-edited model's own rollouts rarely cover it, which limits the effectiveness of pure on-policy distillation. To close this gap, we propose HPSE, which builds a hybrid rollout that steps in to place missing facts onto the student's own trajectory precisely where its coverage fails, while staying on-policy elsewhere. We theoretically analyze HPSE's advantage over pure on-policy distillation, and empirically establish its plug-and-play improvements across four LLM backbones and two KE editors under various scenarios.