Only-IF：揭示指导多样性对泛化的决定性影响

Only-IF:Revealing the Decisive Effect of Instruction Diversity on Generalization

October 7, 2024

作者: Dylan Zhang, Justin Wang, Francois Charton

cs.AI

摘要

理解并准确遵循指令对于大型语言模型（LLMs）在各种任务中发挥有效作用至关重要。在这项工作中，我们严格审查了使模型能够泛化到未见指令的关键因素，为指导指令调整数据的收集提供了见解。通过受图灵完备马尔可夫算法启发的受控实验，我们证明了只有当训练数据在语义领域上足够多样化时，这种泛化才会出现。我们的研究结果还表明，仅在有限领域内进行多样化并不能确保稳健的泛化。相比之下，跨领域数据多样化，即使在受限的数据预算下，也显著增强了模型的适应能力。我们进一步将分析扩展到现实场景，包括对专家和通用模型的微调。在这两种情况下，我们证明了：1）通过增加已建立数据集的多样性，同时保持数据规模恒定，可以实现更好的性能；2）在扩大数据规模时，通过使指令的语义多样化比简单增加类似数据的数量更为有效。我们的研究为数据集整理提供了重要见解，特别是在通过扩展专家和通用场景的训练数据来优化模型性能时。我们表明，仔细考虑数据多样化是关键的：通过使用超出其核心领域的数据来训练专家模型，可以显著提高性能，而通用模型则受益于增强其在各种应用中整体遵循指令能力的多样数据混合。我们的研究结果突出了战略多样化的关键作用，并提供了改善数据质量的明确指导原则。

English

Understanding and accurately following instructions is critical for large language models (LLMs) to be effective across diverse tasks. In this work, we rigorously examine the key factors that enable models to generalize to unseen instructions, providing insights to guide the collection of data for instruction-tuning. Through controlled experiments, inspired by the Turing-complete Markov algorithm, we demonstrate that such generalization only emerges when training data is diversified enough across semantic domains. Our findings also reveal that merely diversifying within limited domains fails to ensure robust generalization. In contrast, cross-domain data diversification, even under constrained data budgets, significantly enhances a model's adaptability. We further extend our analysis to real-world scenarios, including fine-tuning of $textbf{specialist} and textbf{generalist}$ models. In both cases, we demonstrate that 1) better performance can be achieved by increasing the diversity of an established dataset while keeping the data size constant, and 2) when scaling up the data, diversifying the semantics of instructions is more effective than simply increasing the quantity of similar data. Our research provides important insights for dataset collation, particularly when optimizing model performance by expanding training data for both specialist and generalist scenarios. We show that careful consideration of data diversification is key: training specialist models with data extending beyond their core domain leads to significant performance improvements, while generalist models benefit from diverse data mixtures that enhance their overall instruction-following capabilities across a wide range of applications. Our results highlight the critical role of strategic diversification and offer clear guidelines for improving data quality.