噪声指令上的微调：对泛化与性能的影响

摘要

指令微调在提升大语言模型（LLMs）的任务解决能力方面发挥着关键作用，增强了其在各类任务中生成有用响应的实用性。然而，先前的研究表明，LLMs对指令表述的细微变化极为敏感。本文探讨了在指令微调数据中引入扰动是否能增强LLMs对噪声指令的抵抗力。我们重点关注通过去除停用词或打乱词序等扰动方式进行指令微调，如何影响LLMs在原始及扰动版本广泛使用的基准测试（MMLU、BBH、GSM8K）上的表现。此外，我们还评估了学习动态及模型行为的潜在变化。令人惊讶的是，我们的结果表明，在某些情况下，基于扰动指令的指令微调能够提升下游任务性能。这些发现强调了在指令微调中包含扰动指令的重要性，这可以使LLMs对用户输入的噪声更具韧性。

English

Instruction-tuning plays a vital role in enhancing the task-solving abilities of large language models (LLMs), improving their usability in generating helpful responses on various tasks. However, previous work has demonstrated that they are sensitive to minor variations in instruction phrasing. In this paper, we explore whether introducing perturbations in instruction-tuning data can enhance LLMs' resistance against noisy instructions. We focus on how instruction-tuning with perturbations, such as removing stop words or shuffling words, affects LLMs' performance on the original and perturbed versions of widely-used benchmarks (MMLU, BBH, GSM8K). We further assess learning dynamics and potential shifts in model behavior. Surprisingly, our results suggest that instruction-tuning on perturbed instructions can, in some cases, improve downstream performance. These findings highlight the importance of including perturbed instructions in instruction-tuning, which can make LLMs more resilient to noisy user inputs.

噪声指令上的微调：对泛化与性能的影响

Fine-Tuning on Noisy Instructions: Effects on Generalization and Performance

摘要

Support