TCIA：一种面向指令微调的任务中心化指令增强方法

TCIA: A Task-Centric Instruction Augmentation Method for Instruction Finetuning

August 28, 2025

作者: Simin Ma, Shujian Liu, Jun Tan, Yebowen Hu, Song Wang, Sathish Reddy Indurthi, Sanqiang Zhao, Liwei Wu, Jianbing Han, Kaiqiang Song

cs.AI

摘要

多样化的指令数据对于大型语言模型的有效指令调优至关重要，因为它使模型能够泛化处理不同类型的输入。构建这种多样化的指令数据集是这一过程中的关键步骤。现有方法通常利用大型语言模型自动探索并生成多样化的指令，确保数据的多样性和质量。然而，这些方法往往忽视了实际应用中的一个重要因素：任务相关性。实际上，只有少数现实应用需要真正通用的模型；大多数应用则受益于针对其特定用例量身定制的任务相关知识。因此，开发既保持多样性又针对具体现实场景优化的指令增强方法显得尤为重要。 为此，我们引入了任务中心化指令增强（Task Centric Instruction Augmentation, TCIA）框架，该系统在保持多样性的同时，确保指令与任务的对齐。通过在离散的查询-约束空间中表示指令，TCIA生成了一组丰富的任务相关指令，使模型能够在不牺牲整体性能的情况下，泛化到这些特定任务的指令。实验表明，TCIA在四个现实世界的任务特定应用中，平均提升了开源大型语言模型8.7%的性能，在某些情况下甚至超越了领先的闭源模型。这些改进并未削弱模型的一般指令遵循能力，使得TCIA成为将大型语言模型适配于现实世界、任务导向应用的可扩展且高效的解决方案。

English

Diverse instruction data is vital for effective instruction tuning of large language models, as it enables the model to generalize across different types of inputs . Building such diversified instruction dataset is an essential step in this process. Existing approaches often leverage large language models to automatically explore and generate diverse instructions, ensuring both data diversity and quality. However, they tend to overlook an important factor in real-world applications: on-task relevance. In practice, only a few real-world applications require a truly general-purpose model; most benefit from task-specific knowledge tailored to their particular use case. Therefore, it is vital to develop instruction augmentation methods that not only maintain diversity but are also optimized for specific, real-world scenarios. We thus introduce Task Centric Instruction Augmentation (TCIA), a framework that systematically expands instructions while preserving both diversity and task alignment. By representing instructions in a discrete query-constraints space, TCIA creates a rich set of task-relevant instructions and enables models to generalize to these task-specific instructions without sacrificing overall performance. Experiments show that TCIA improves open-source LLMs' performance by an average of 8.7% across four real-world, task-specific applications, and in some cases outperforming leading closed-source models. These improvements do not compromise general instruction-following ability, making TCIA a scalable and efficient solution for adapting LLMs to real-world, task-focused applications.