搜索指令：通过基于检索的指令数据集创建增强领域适应性

SearchInstruct: Enhancing Domain Adaptation via Retrieval-Based Instruction Dataset Creation

September 12, 2025

作者: Iman Barati, Mostafa Amiri, Heshaam Faili

cs.AI

摘要

监督微调（SFT）对于训练大型语言模型（LLMs）至关重要，它能显著提升诸如指令遵循和上下文学习等关键能力。然而，由于特定领域的独特限制和数据稀缺性，创建适用于这些领域的高质量训练数据集仍面临挑战。本文提出了一种创新方法——SearchInstruct，专门用于构建高质量的SFT指令数据集。该方法始于一组有限的、由人工生成的领域特定问题，随后利用大型语言模型系统性地扩展这些问题。接着，动态检索领域相关资源，为每个扩展问题生成准确且上下文恰当的答案。实验评估表明，SearchInstruct不仅提升了SFT数据集的多样性和质量，还带来了LLMs在特定领域性能的显著提升。此外，我们展示了该方法在数据集生成之外，还能有效支持模型编辑等任务，实现对现有模型的高效更新。为了促进研究的可重复性和社区采用，我们在公开的Git仓库中提供了完整的实现细节、生成的指令-响应对全集以及源代码：[https://github.com/mostafaamiri/SearchInstruct](https://github.com/mostafaamiri/SearchInstruct)。

English

Supervised Fine-Tuning (SFT) is essential for training large language models (LLMs), significantly enhancing critical capabilities such as instruction following and in-context learning. Nevertheless, creating suitable training datasets tailored for specific domains remains challenging due to unique domain constraints and data scarcity. In this paper, we propose SearchInstruct, an innovative method explicitly designed to construct high quality instruction datasets for SFT. Our approach begins with a limited set of domain specific, human generated questions, which are systematically expanded using a large language model. Subsequently, domain relevant resources are dynamically retrieved to generate accurate and contextually appropriate answers for each augmented question. Experimental evaluation demonstrates that SearchInstruct enhances both the diversity and quality of SFT datasets, leading to measurable improvements in LLM performance within specialized domains. Additionally, we show that beyond dataset generation, the proposed method can also effectively facilitate tasks such as model editing, enabling efficient updates to existing models. To facilitate reproducibility and community adoption, we provide full implementation details, the complete set of generated instruction response pairs, and the source code in a publicly accessible Git repository: [https://github.com/mostafaamiri/SearchInstruct](https://github.com/mostafaamiri/SearchInstruct)