CaKE：電路感知編輯實現可泛化的知識學習器

CaKE: Circuit-aware Editing Enables Generalizable Knowledge Learners

March 20, 2025

作者: Yunzhi Yao, Jizhan Fang, Jia-Chen Gu, Ningyu Zhang, Shumin Deng, Huajun Chen, Nanyun Peng

cs.AI

摘要

知識編輯（Knowledge Editing, KE）使得我們能夠修改大型語言模型（LLMs）中過時或錯誤的資訊。雖然現有的KE方法可以更新孤立的事實，但這些方法在將更新推廣到依賴於修改後知識的多跳推理任務時卻面臨困難。通過分析推理迴路——LLMs用於基於知識推理的神經路徑，我們觀察到當前層局部化的KE方法，如MEMIT和WISE，這些方法僅編輯單個或少數模型層，難以有效地將更新後的資訊整合到這些推理路徑中。為了解決這一限制，我們提出了CaKE（Circuit-aware Knowledge Editing），這是一種新穎的方法，能夠更有效地將更新後的知識整合到LLMs中。CaKE利用基於迴路分析策略性策劃的數據，迫使模型利用修改後的知識，激勵模型為新整合的知識開發適當的推理迴路。實驗結果顯示，CaKE在相關推理任務中能夠更準確和一致地使用更新後的知識，與現有的KE方法相比，在MQuAKE數據集上的多跳推理準確率平均提高了20%。我們在https://github.com/zjunlp/CaKE上發布了代碼和數據。

English

Knowledge Editing (KE) enables the modification of outdated or incorrect information in large language models (LLMs). While existing KE methods can update isolated facts, they struggle to generalize these updates to multi-hop reasoning tasks that depend on the modified knowledge. Through an analysis of reasoning circuits -- the neural pathways LLMs use for knowledge-based inference, we observe that current layer-localized KE approaches, such as MEMIT and WISE, which edit only single or a few model layers, struggle to effectively incorporate updated information into these reasoning pathways. To address this limitation, we propose CaKE (Circuit-aware Knowledge Editing), a novel method that enables more effective integration of updated knowledge in LLMs. CaKE leverages strategically curated data, guided by our circuits-based analysis, that enforces the model to utilize the modified knowledge, stimulating the model to develop appropriate reasoning circuits for newly integrated knowledge. Experimental results show that CaKE enables more accurate and consistent use of updated knowledge across related reasoning tasks, leading to an average of 20% improvement in multi-hop reasoning accuracy on MQuAKE dataset compared to existing KE methods. We release the code and data in https://github.com/zjunlp/CaKE.