大语言模型智能体能按剧本行事吗？——面向交互式叙事长时程一致性的评测基准

摘要

大型语言模型（LLMs）的快速发展正在通过实现开放式、流畅的交互式叙事，彻底改变游戏人工智能领域。然而，现有研究在很大程度上忽视了在不受约束的用户干预下，维持长期逻辑一致性与叙事完整性的关键挑战。为解决这一问题，我们将该挑战形式化为叙事承诺保持（Narrative Commitment Preservation, NCP），并以交互式叙事作为测试平台。我们提出了NCP-Bench，一个包含100个源自电影梗概的叙事环境的基准测试集。每个环境都包含一个结构化的叙事规范（轨迹、承诺和初始事实），我们可以在玩家智能体与叙述者智能体的整个交互过程中对其进行自动检查。跨最先进LLMs的实验揭示了一个显著的长期一致性差距：高语言质量并不能保证承诺的保持；即使是强大的模型，在对抗性干预下也频繁生成逻辑冲突的内容，其中表现最佳的模型（GPT-5.2）在20轮后的存活率仅为42%，各模型的事实冲突率介于40%至68%之间，且仅有个别运行能在100轮限制内满足所有成就承诺。

English

The rapid advancement of Large Language Models (LLMs) is revolutionizing AI for Games by enabling open-ended and fluid interactive storytelling. However, existing research has largely overlooked the critical challenge of maintaining long-horizon logical consistency and narrative integrity against unconstrained user interventions. To address this, we formulate this challenge as Narrative Commitment Preservation (NCP), and take interactive narrative as our testbed. We introduce NCP-Bench, a benchmark of 100 narrative environments derived from movie synopses. Each environment includes a structured narrative specification (trajectory, commitments, and initial facts) that we can automatically check throughout the interaction between the player agent and the narrator agent. Experiments across state-of-the-art LLMs reveal a substantial long-horizon consistency gap: high linguistic quality does not guarantee commitment preservation; even strong models frequently generate logically conflicting content under adversarial interventions, with the best-performing model (GPT-5.2) achieving only 42% survival rate after 20 turns and fact conflict rates ranging from 40% to 68% across models, and only isolated runs satisfying all achievement commitments within the 100-turn limit.