ToMAP：运用心智理论训练具备对手意识的LLM说服者

ToMAP: Training Opponent-Aware LLM Persuaders with Theory of Mind

May 29, 2025

作者: Peixuan Han, Zijia Liu, Jiaxuan You

cs.AI

摘要

大型语言模型（LLMs）在说服任务中展现出显著潜力，然而现有关于训练LLM说服者的研究仍处于初步阶段。值得注意的是，人类擅长主动且动态地模拟对手的思维与观点，而当前的LLMs在此类心智理论（Theory of Mind, ToM）推理上表现欠佳，导致其多样性与对手意识受限。为克服这一局限，我们提出了心智理论增强型说服者（Theory of Mind Augmented Persuader, ToMAP），这是一种通过整合两个心智理论模块来提升说服者对对手心理状态的意识与分析能力的新颖方法。具体而言，我们首先提示说服者考虑目标核心主张可能遭遇的反对意见，随后利用文本编码器配合训练好的多层感知机分类器预测对手对这些反论点的当前立场。我们精心设计的强化学习框架使说服者学会分析对手相关信息，并运用这些信息生成更具说服力的论点。实验表明，仅含30亿参数的ToMAP说服者在多个被说服模型及多样语料库上的表现均优于如GPT-4o等更大规模的基线模型，相对提升达39.4%。尤为突出的是，ToMAP在训练过程中展现出复杂的推理链条和减少的重复现象，从而产生更多样且有效的论点。ToMAP的对手意识特性还使其适用于长时间对话，并能采用更具逻辑性和对手意识的策略。这些结果验证了我们方法的有效性，并凸显了其在开发更具说服力的语言代理方面的潜力。代码已发布于：https://github.com/ulab-uiuc/ToMAP。

English

Large language models (LLMs) have shown promising potential in persuasion, but existing works on training LLM persuaders are still preliminary. Notably, while humans are skilled in modeling their opponent's thoughts and opinions proactively and dynamically, current LLMs struggle with such Theory of Mind (ToM) reasoning, resulting in limited diversity and opponent awareness. To address this limitation, we introduce Theory of Mind Augmented Persuader (ToMAP), a novel approach for building more flexible persuader agents by incorporating two theory of mind modules that enhance the persuader's awareness and analysis of the opponent's mental state. Specifically, we begin by prompting the persuader to consider possible objections to the target central claim, and then use a text encoder paired with a trained MLP classifier to predict the opponent's current stance on these counterclaims. Our carefully designed reinforcement learning schema enables the persuader learns how to analyze opponent-related information and utilize it to generate more effective arguments. Experiments show that the ToMAP persuader, while containing only 3B parameters, outperforms much larger baselines, like GPT-4o, with a relative gain of 39.4% across multiple persuadee models and diverse corpora. Notably, ToMAP exhibits complex reasoning chains and reduced repetition during training, which leads to more diverse and effective arguments. The opponent-aware feature of ToMAP also makes it suitable for long conversations and enables it to employ more logical and opponent-aware strategies. These results underscore our method's effectiveness and highlight its potential for developing more persuasive language agents. Code is available at: https://github.com/ulab-uiuc/ToMAP.