PersonaX：基於大型語言模型推斷行為特徵的多模態數據集

摘要

理解人類行為特質是人機交互、計算社會科學和個性化人工智能系統應用的核心。這種理解通常需要整合多種模態來捕捉細微的模式和關係。然而，現有資源很少提供將行為描述符與面部屬性和傳記信息等互補模態相結合的數據集。為填補這一空白，我們提出了PersonaX，這是一個精心策劃的多模態數據集集合，旨在實現跨模態的公共特質全面分析。PersonaX包含兩部分：(1) CelebPersona，涵蓋了來自不同職業的9444位公眾人物；(2) AthlePersona，覆蓋了7大主要體育聯盟的4181名職業運動員。每個數據集都包括由三個高性能大型語言模型推斷的行為特質評估，以及面部圖像和結構化的傳記特徵。我們從兩個互補層面分析PersonaX。首先，我們從文本描述中抽象出高層次特質分數，並應用五種統計獨立性檢驗來探討它們與其他模態的關係。其次，我們引入了一種新穎的因果表示學習（CRL）框架，專為多模態和多測量數據設計，提供了理論上的可識別性保證。在合成數據和真實世界數據上的實驗證明了我們方法的有效性。通過統一結構化和非結構化分析，PersonaX為研究與視覺和傳記屬性相結合的LLM推斷行為特質奠定了基礎，推動了多模態特質分析和因果推理的發展。

English

Understanding human behavior traits is central to applications in human-computer interaction, computational social science, and personalized AI systems. Such understanding often requires integrating multiple modalities to capture nuanced patterns and relationships. However, existing resources rarely provide datasets that combine behavioral descriptors with complementary modalities such as facial attributes and biographical information. To address this gap, we present PersonaX, a curated collection of multimodal datasets designed to enable comprehensive analysis of public traits across modalities. PersonaX consists of (1) CelebPersona, featuring 9444 public figures from diverse occupations, and (2) AthlePersona, covering 4181 professional athletes across 7 major sports leagues. Each dataset includes behavioral trait assessments inferred by three high-performing large language models, alongside facial imagery and structured biographical features. We analyze PersonaX at two complementary levels. First, we abstract high-level trait scores from text descriptions and apply five statistical independence tests to examine their relationships with other modalities. Second, we introduce a novel causal representation learning (CRL) framework tailored to multimodal and multi-measurement data, providing theoretical identifiability guarantees. Experiments on both synthetic and real-world data demonstrate the effectiveness of our approach. By unifying structured and unstructured analysis, PersonaX establishes a foundation for studying LLM-inferred behavioral traits in conjunction with visual and biographical attributes, advancing multimodal trait analysis and causal reasoning.

PersonaX：基於大型語言模型推斷行為特徵的多模態數據集

PersonaX: Multimodal Datasets with LLM-Inferred Behavior Traits

摘要

Support