唯-IF：揭示指導多樣性對泛化的決定性影響

Only-IF:Revealing the Decisive Effect of Instruction Diversity on Generalization

October 7, 2024

作者: Dylan Zhang, Justin Wang, Francois Charton

cs.AI

摘要

為了使大型語言模型（LLMs）在各種任務中發揮效力，理解並準確遵循指示至關重要。在這項工作中，我們嚴謹地研究了使模型能夠推廣到未見指示的關鍵因素，提供了指導收集指示調整數據的見解。通過受圖靈完備馬爾可夫算法啟發的受控實驗，我們證明當訓練數據在語義領域之間足夠多樣化時，這種泛化只有在這種情況下才會出現。我們的研究結果還顯示，僅在有限領域內進行多樣化無法確保堅固的泛化。相反，跨領域數據多樣化，即使在受限的數據預算下，也顯著增強了模型的適應能力。我們進一步將分析擴展到現實情境，包括對專家和通用模型的微調。在這兩種情況下，我們證明了以下兩點：1）通過增加已建立數據集的多樣性，同時保持數據大小不變，可以實現更好的性能；2）當擴大數據時，多樣化指示的語義比單純增加相似數據的數量更有效。我們的研究為數據集整理提供了重要見解，特別是在通過擴展專家和通用情境的訓練數據來優化模型性能時。我們表明，仔細考慮數據多樣化是關鍵的：用超出其核心領域的數據訓練專家模型將帶來顯著的性能改進，而通用模型則受益於增強其在各種應用中整體遵循指示能力的多樣數據組合。我們的結果突顯了戰略多樣化的關鍵作用，並提供了改善數據質量的明確指南。

English

Understanding and accurately following instructions is critical for large language models (LLMs) to be effective across diverse tasks. In this work, we rigorously examine the key factors that enable models to generalize to unseen instructions, providing insights to guide the collection of data for instruction-tuning. Through controlled experiments, inspired by the Turing-complete Markov algorithm, we demonstrate that such generalization only emerges when training data is diversified enough across semantic domains. Our findings also reveal that merely diversifying within limited domains fails to ensure robust generalization. In contrast, cross-domain data diversification, even under constrained data budgets, significantly enhances a model's adaptability. We further extend our analysis to real-world scenarios, including fine-tuning of $textbf{specialist} and textbf{generalist}$ models. In both cases, we demonstrate that 1) better performance can be achieved by increasing the diversity of an established dataset while keeping the data size constant, and 2) when scaling up the data, diversifying the semantics of instructions is more effective than simply increasing the quantity of similar data. Our research provides important insights for dataset collation, particularly when optimizing model performance by expanding training data for both specialist and generalist scenarios. We show that careful consideration of data diversification is key: training specialist models with data extending beyond their core domain leads to significant performance improvements, while generalist models benefit from diverse data mixtures that enhance their overall instruction-following capabilities across a wide range of applications. Our results highlight the critical role of strategic diversification and offer clear guidelines for improving data quality.