使用大型語言模型建構社會世界模型

摘要

理解並預測社會信念如何因應事件（從政策變革到科學突破）而演變，仍是社會科學中的一項基本挑戰。基於大型語言模型的常識知識與社交智慧，我們提出疑問：大型語言模型能否模擬社會事件後社會信念的動態變化？在本研究中，我們引入「社會世界模型」概念，這是一個通用框架，旨在捕捉社會信念如何因重大事件而演變。社會世界模型透過挖掘社會資料中的時間模式並優化證據下界來學習社會信念的狀態轉移函數，無需仰賴連結事件與信念變遷的人工標註，亦無需昂貴的普查資料。為評估社會世界模型，我們提出一個基準測試——「SWM-bench」，該基準源自真實世界的預測市場，特別是 Kalshi 與 Polymarket。SWM-bench 包含超過 12,000 筆資料點，涵蓋政治、金融與加密貨幣等多元領域的社會信念預測任務。實驗結果顯示，社會世界模型顯著優於時間序列基礎模型，在 Kalshi 資料上達到最先進成果，並在 Polymarket 資料上展現競爭力，同時為社會信念動態的潛在機制提供可解釋的洞見。

English

Understanding and predicting how social beliefs evolve in response to events -- from policy changes to scientific breakthroughs -- remains a fundamental challenge in social science. Given LLMs' commonsense knowledge and social intelligence, we ask: Can LLMs model the dynamics of social beliefs following social events? In this work, we introduce the concept of the Social World Model (SWM), a general framework designed to capture how social beliefs evolve in response to major events. SWM learns state-transition functions for social beliefs by mining temporal patterns in social data and optimizing the evidence lower bound, without the need for explicit human annotations linking events to belief shifts, or for expensive census data. To evaluate SWM, we introduce a benchmark, SWM-bench, derived from real-world prediction markets, specifically Kalshi and Polymarket. SWM-bench includes over 12k data points for social belief prediction tasks spanning diverse domains such as politics, finance, and cryptocurrency. Our experimental results show that SWM significantly outperforms time-series foundation models, achieving state-of-the-art results on Kalshi data and demonstrating competitive performance on Polymarket data, while offering interpretable insights into the underlying mechanisms of social belief dynamics.