ToMAP：運用心智理論訓練具備對手意識的大型語言模型說服者

ToMAP: Training Opponent-Aware LLM Persuaders with Theory of Mind

May 29, 2025

作者: Peixuan Han, Zijia Liu, Jiaxuan You

cs.AI

摘要

大型語言模型（LLMs）在說服方面展現了顯著的潛力，但現有針對訓練LLM說服者的研究仍處於初步階段。值得注意的是，儘管人類擅長主動且動態地模擬對手的思維和觀點，目前的LLMs在這種心智理論（Theory of Mind, ToM）推理上仍面臨挑戰，導致其多樣性和對手意識有限。為解決這一限制，我們引入了心智理論增強型說服者（Theory of Mind Augmented Persuader, ToMAP），這是一種新穎的方法，通過整合兩個心智理論模組來增強說服者對對手心理狀態的意識和分析能力。具體而言，我們首先提示說服者考慮對目標核心主張可能的反對意見，然後使用文本編碼器配以訓練過的多層感知器（MLP）分類器來預測對手對這些反駁觀點的當前立場。我們精心設計的強化學習框架使說服者學會如何分析與對手相關的信息，並利用這些信息生成更有效的論點。實驗表明，僅包含30億參數的ToMAP說服者在多種說服對象模型和多樣化的語料庫上，相較於如GPT-4o等更大規模的基線模型，取得了39.4%的相對增益。值得注意的是，ToMAP在訓練過程中展現了複雜的推理鏈條並減少了重複，從而產生了更多樣且有效的論點。ToMAP的對手意識特性也使其適合於長時間對話，並能運用更具邏輯性和對手意識的策略。這些結果證明了我們方法的有效性，並凸顯了其在開發更具說服力的語言代理方面的潛力。代碼可於以下網址獲取：https://github.com/ulab-uiuc/ToMAP。

English

Large language models (LLMs) have shown promising potential in persuasion, but existing works on training LLM persuaders are still preliminary. Notably, while humans are skilled in modeling their opponent's thoughts and opinions proactively and dynamically, current LLMs struggle with such Theory of Mind (ToM) reasoning, resulting in limited diversity and opponent awareness. To address this limitation, we introduce Theory of Mind Augmented Persuader (ToMAP), a novel approach for building more flexible persuader agents by incorporating two theory of mind modules that enhance the persuader's awareness and analysis of the opponent's mental state. Specifically, we begin by prompting the persuader to consider possible objections to the target central claim, and then use a text encoder paired with a trained MLP classifier to predict the opponent's current stance on these counterclaims. Our carefully designed reinforcement learning schema enables the persuader learns how to analyze opponent-related information and utilize it to generate more effective arguments. Experiments show that the ToMAP persuader, while containing only 3B parameters, outperforms much larger baselines, like GPT-4o, with a relative gain of 39.4% across multiple persuadee models and diverse corpora. Notably, ToMAP exhibits complex reasoning chains and reduced repetition during training, which leads to more diverse and effective arguments. The opponent-aware feature of ToMAP also makes it suitable for long conversations and enables it to employ more logical and opponent-aware strategies. These results underscore our method's effectiveness and highlight its potential for developing more persuasive language agents. Code is available at: https://github.com/ulab-uiuc/ToMAP.