TCIA：一種面向指令微調的任務中心化指令增強方法

TCIA: A Task-Centric Instruction Augmentation Method for Instruction Finetuning

August 28, 2025

作者: Simin Ma, Shujian Liu, Jun Tan, Yebowen Hu, Song Wang, Sathish Reddy Indurthi, Sanqiang Zhao, Liwei Wu, Jianbing Han, Kaiqiang Song

cs.AI

摘要

多樣化的指令數據對於大型語言模型的有效指令調優至關重要，因為它使模型能夠泛化到不同類型的輸入。構建這樣的多樣化指令數據集是這一過程中的關鍵步驟。現有方法通常利用大型語言模型自動探索和生成多樣化的指令，確保數據的多樣性和質量。然而，這些方法往往忽視了實際應用中的一個重要因素：任務相關性。在實踐中，只有少數實際應用需要真正通用的模型；大多數應用則受益於針對其特定用例量身定制的任務特定知識。因此，開發既保持多樣性又針對特定實際場景進行優化的指令增強方法至關重要。 為此，我們引入了任務中心指令增強（Task Centric Instruction Augmentation, TCIA）框架，該框架在保持多樣性和任務對齊的同時，系統性地擴展指令。通過在離散的查詢-約束空間中表示指令，TCIA 創建了一組豐富的任務相關指令，使模型能夠在不犧牲整體性能的情況下泛化到這些任務特定指令。實驗表明，TCIA 在四個實際的任務特定應用中，將開源大型語言模型的性能平均提高了 8.7%，在某些情況下甚至超越了領先的閉源模型。這些改進並未損害一般的指令遵循能力，使得 TCIA 成為一種可擴展且高效的解決方案，用於使大型語言模型適應實際的任務導向應用。

English

Diverse instruction data is vital for effective instruction tuning of large language models, as it enables the model to generalize across different types of inputs . Building such diversified instruction dataset is an essential step in this process. Existing approaches often leverage large language models to automatically explore and generate diverse instructions, ensuring both data diversity and quality. However, they tend to overlook an important factor in real-world applications: on-task relevance. In practice, only a few real-world applications require a truly general-purpose model; most benefit from task-specific knowledge tailored to their particular use case. Therefore, it is vital to develop instruction augmentation methods that not only maintain diversity but are also optimized for specific, real-world scenarios. We thus introduce Task Centric Instruction Augmentation (TCIA), a framework that systematically expands instructions while preserving both diversity and task alignment. By representing instructions in a discrete query-constraints space, TCIA creates a rich set of task-relevant instructions and enables models to generalize to these task-specific instructions without sacrificing overall performance. Experiments show that TCIA improves open-source LLMs' performance by an average of 8.7% across four real-world, task-specific applications, and in some cases outperforming leading closed-source models. These improvements do not compromise general instruction-following ability, making TCIA a scalable and efficient solution for adapting LLMs to real-world, task-focused applications.