通过先决条件学习，虚构的合成数据可以提高LLM事实性。

Fictitious Synthetic Data Can Improve LLM Factuality via Prerequisite Learning

October 25, 2024

作者: Yujian Liu, Shiyu Chang, Tommi Jaakkola, Yang Zhang

cs.AI

摘要

最近的研究发现，LLM幻觉的一个加剧因素是预训练和微调之间的知识不一致，即不熟悉的微调数据会误导LLM制造出似是而非的错误输出。在本文中，我们提出了一种名为Prereq-Tune的新颖微调策略，以解决这种知识不一致性并减少幻觉。从根本上讲，Prereq-Tune将技能和知识的学习分离，使模型仅学习任务技能而不受知识不一致性的影响。为实现这一目标，Prereq-Tune引入了一个额外的先决学习阶段，用于学习SFT所需的知识，从而使后续的SFT仅专注于任务技能。Prereq-Tune还可以与虚构的合成数据结合，以增强LLM输出与其内部知识的联系。实验证明，Prereq-Tune在提高LLM在短问答和长篇生成任务中的事实性方面优于现有基线。它还为LLM中的知识受控生成开辟了新的可能性。我们的代码可在https://github.com/UCSB-NLP-Chang/Prereq_tune.git找到。

English

Recent studies have identified one aggravating factor of LLM hallucinations as the knowledge inconsistency between pre-training and fine-tuning, where unfamiliar fine-tuning data mislead the LLM to fabricate plausible but wrong outputs. In this paper, we propose a novel fine-tuning strategy called Prereq-Tune to address this knowledge inconsistency and reduce hallucinations. Fundamentally, Prereq-Tune disentangles the learning of skills and knowledge, so the model learns only the task skills without being impacted by the knowledge inconsistency. To achieve this, Prereq-Tune introduces an additional prerequisite learning stage to learn the necessary knowledge for SFT, allowing subsequent SFT to focus only on task skills. Prereq-Tune can also be combined with fictitious synthetic data to enhance the grounding of LLM outputs to their internal knowledge. Experiments show that Prereq-Tune outperforms existing baselines in improving LLM's factuality across short QA and long-form generation tasks. It also opens new possibilities for knowledge-controlled generation in LLMs. Our code is available at https://github.com/UCSB-NLP-Chang/Prereq_tune.git.