PreScam：一個用於從早期對話預測詐騙進展的基準

摘要

對話式詐騙，例如愛情詐騙與投資詐騙，正逐漸成為網路詐騙的主要形式。不同於假中獎或未繳通行費訊息等一次性詐騙誘餌，這類詐騙透過多輪對話展開，詐騙者利用不斷演進的心理操弄手法，逐步控制受害者。然而，現有研究主要聚焦於靜態詐騙偵測或合成詐騙案例，對於語言模型是否能理解真實世界詐騙隨時間推移的演進過程，仍屬未知。我們提出 PreScam，一個用於模擬詐騙從早期對話開始演進的基準測試。PreScam 基於使用者提交的詐騙回報建構，從 177,989 筆原始回報中篩選並整理出 11,573 個對話式詐騙案例，涵蓋 20 種詐騙類別。每個案例根據所提出的詐騙殺傷鏈所定義的詐騙生命週期進行階層式架構，並進一步在對話輪次層級上，對詐騙者的心理行動與受害者回應進行標註。我們在兩項任務上進行模型基準測試：即時終止預測（評估對話是否接近終止階段）與詐騙者行動預測（預測詐騙者接下來將採取的動作）。結果顯示，表面流暢度與進程建模之間存在明顯落差：在即時終止預測方面，監督式編碼器的表現遠優於零樣本大型語言模型；而即使是表現優異的大型語言模型，在下一行動預測上的成效也僅屬中等。綜合來看，這些結果顯示現有模型雖能捕捉部分詐騙相關線索，但在追蹤風險如何逐步升高、以及操弄手法如何在多輪對話中逐漸展開方面，仍有困難。

English

Conversational scams, such as romance and investment scams, are emerging as a major form of online fraud. Unlike one-shot scam lures such as fake lottery or unpaid toll messages, they unfold through multi-turn conversations in which scammers gradually manipulate victims using evolving psychological techniques. However, existing research mainly focuses on static scam detection or synthetic scams, leaving open whether language models can understand how real-world scams progress over time. We introduce PreScam, a benchmark for modeling scam progression from early conversations. Built from user-submitted scam reports, PreScam filters and structures 177,989 raw reports into 11,573 conversational scam instances spanning 20 scam categories. Each instance is hierarchically structured according to the scam lifecycle defined by the proposed scam kill chain, and further annotated at the turn level with scammer psychological actions and victim responses. We benchmark models on two tasks: real-time termination prediction, which estimates whether a conversation is approaching the termination stage, and scammer action prediction, which forecasts the scammer's subsequent actions. Results show a clear gap between surface-level fluency and progression modeling: supervised encoders substantially outperform zero-shot LLMs on real-time termination prediction, while next-action prediction remains only moderately successful even for strong LLMs. Taken together, these results show that current models can capture some scam-related cues, yet still struggle to track how risk escalates and how manipulation unfolds across turns.