ProFit：通过概率引导的令牌选择利用SFT中的高价值信号

ProFit: Leveraging High-Value Signals in SFT via Probability-Guided Token Selection

January 14, 2026

作者: Tao Liu, Taiqiang Wu, Runming Yang, Shaoning Sun, Junjie Wang, Yujiu Yang

cs.AI

摘要

监督微调（SFT）是使大语言模型（LLMs）与人类意图对齐的关键后训练策略。然而，传统SFT常因强制模型对齐单一参考答案而忽略语言的一对多特性，导致模型过度拟合非核心表达。尽管实证分析表明引入多参考答案可缓解此问题，但高昂的数据与计算成本要求我们进行策略性转变：将重点从追求答案多样性转向优先解决单参考过拟合。为实现这一目标，我们揭示了词元概率与语义重要性之间的内在联系：高概率词元承载核心逻辑框架，而低概率词元多为可替换表达。基于此发现，我们提出ProFit方法，通过选择性掩码低概率词元来防止模型陷入表层过拟合。大量实验证实，ProFit在通用推理和数学基准测试中均稳定优于传统SFT基线方法。

English

Supervised fine-tuning (SFT) is a fundamental post-training strategy to align Large Language Models (LLMs) with human intent. However, traditional SFT often ignores the one-to-many nature of language by forcing alignment with a single reference answer, leading to the model overfitting to non-core expressions. Although our empirical analysis suggests that introducing multiple reference answers can mitigate this issue, the prohibitive data and computational costs necessitate a strategic shift: prioritizing the mitigation of single-reference overfitting over the costly pursuit of answer diversity. To achieve this, we reveal the intrinsic connection between token probability and semantic importance: high-probability tokens carry the core logical framework, while low-probability tokens are mostly replaceable expressions. Based on this insight, we propose ProFit, which selectively masks low-probability tokens to prevent surface-level overfitting. Extensive experiments confirm that ProFit consistently outperforms traditional SFT baselines on general reasoning and mathematical benchmarks.