超越单一世界：多元宇宙情境下角色扮演超级英雄的基准测试

摘要

大型语言模型（LLMs）正日益被用作角色扮演代理，然而它们在忠实且一致地演绎特定版本角色——例如跨越漫画与电影宇宙的超级英雄——方面的能力仍待深入探索。漫威与DC等超级英雄经典作品为此提供了丰富的试验场：数十年的故事叙述孕育了同一角色的多个化身，各自拥有独特的历史、价值观及道德准则。为研究此问题，我们推出了“超越单一世界”基准，涵盖30位标志性英雄及其90个特定版本的角色扮演任务。该基准包含两项任务：(i) 经典事件，考察对关键人生阶段的事实回忆；(ii) 道德困境，让模型面对伦理挑战场景。我们依据一个框架对回答进行评分，该框架区分了内部思考（“思考”）与外部决策（“行动”），并进一步提出了“思行一致”指标，量化理由与行动之间的契合度，作为模型可信度的代理。在推理导向与非推理导向模型上的实验得出三点发现：(1) 思维链提示能提升较弱模型的叙事连贯性，但可能削弱较强模型的经典准确性；(2) 同一角色跨版本泛化仍是主要障碍；(3) 模型往往擅长思考或行动之一，却鲜少两者兼备。“超越单一世界”揭示了多元宇宙一致性与推理对齐中的关键缺口，为角色扮演型LLMs提供了一个极具挑战性的评估平台。

English

Large language models (LLMs) are increasingly used as role-playing agents, yet their capacity to faithfully and consistently portray version-specific characters -- for example, superheroes across comic and cinematic universes -- remains underexplored. Superhero canons such as Marvel and DC provide a rich testbed: decades of storytelling yield multiple incarnations of the same character with distinct histories, values, and moral codes. To study this problem, we introduce Beyond One World, a benchmark for character-grounded roleplay spanning 30 iconic heroes and 90 canon-specific versions. The benchmark comprises two tasks: (i) Canon Events, which probes factual recall of pivotal life stages, and (ii) Moral Dilemmas, which confronts models with ethically charged scenarios. We score responses for canonical accuracy and reasoning fidelity under a framework that separates internal deliberation ("thinking") from outward decisions ("acting"). We further propose Think-Act Matching, a metric that quantifies alignment between reasons and actions and serves as a proxy for model trustworthiness. Experiments across reasoning- and non-reasoning-oriented models yield three findings: (1) chain-of-thought prompting improves narrative coherence in weaker models but can reduce canonical accuracy in stronger ones; (2) cross-version generalization within a character remains a major obstacle; and (3) models often excel at either thinking or acting, but rarely both. Beyond One World exposes critical gaps in multiversal consistency and reasoning alignment, offering a challenging evaluation for role-playing LLMs.

超越单一世界：多元宇宙情境下角色扮演超级英雄的基准测试

Beyond One World: Benchmarking Super Heros in Role-Playing Across Multiversal Contexts

摘要

Support