PAFT：提示無關的微調

PAFT: Prompt-Agnostic Fine-Tuning

February 18, 2025

作者: Chenxing Wei, Yao Shu, Mingwen Ou, Ying Tiffany He, Fei Richard Yu

cs.AI

摘要

儘管大型語言模型（LLMs）在微調後能良好適應下游任務，這種適應性往往會犧牲提示的穩健性，因為即使提示的微小變化也可能顯著降低模型表現。為解決此問題，我們提出了提示無關微調（Prompt-Agnostic Fine-Tuning, PAFT），這是一種簡單而有效的方法，能在微調過程中動態調整提示。此方法促使模型學習任務的基礎原則，而非過度擬合特定的提示表述。PAFT分為兩個階段：首先，構建一組多樣且有意義的合成候選提示；其次，在微調過程中，從這組提示中隨機抽樣，以創建動態的訓練輸入。跨多種數據集和LLMs的廣泛實驗表明，使用PAFT訓練的模型在面對各種提示（包括未見過的提示）時，展現出強大的穩健性和泛化能力。這種增強的穩健性不僅提升了模型表現和推理速度，同時保持了訓練效率。消融研究進一步證實了PAFT的有效性。

English

While Large Language Models (LLMs) adapt well to downstream tasks after fine-tuning, this adaptability often compromises prompt robustness, as even minor prompt variations can significantly degrade performance. To address this, we propose Prompt-Agnostic Fine-Tuning(PAFT), a simple yet effective approach that dynamically adjusts prompts during fine-tuning. This encourages the model to learn underlying task principles rather than overfitting to specific prompt formulations. PAFT operates in two stages: First, a diverse set of meaningful, synthetic candidate prompts is constructed. Second, during fine-tuning, prompts are randomly sampled from this set to create dynamic training inputs. Extensive experiments across diverse datasets and LLMs demonstrate that models trained with PAFT exhibit strong robustness and generalization across a wide range of prompts, including unseen ones. This enhanced robustness improves both model performance and inference speed while maintaining training efficiency. Ablation studies further confirm the effectiveness of PAFT.