EvoGenUI-Bench：评估大语言模型作为多轮生成式UI助手

摘要

大语言模型能够生成交互式网页界面，但可靠的生成式用户界面要求在用户请求不断演化时保持可执行工件。我们引入EvoGenUI-Bench，一个用于多轮界面维护的基准测试，包含150个五轮任务和750个轮次，涵盖三种场景：信息呈现、可执行交互和基于工具的外部状态。我们在浏览器中执行生成的工件，并使用截图、源码和DOM证据、执行器轨迹以及运行时日志进行评估。除了轮次层面和会话层面的成功率，我们还使用相邻通过保留率来衡量跨轮次保持性。在八个模型中，即使最强的模型也仅达到74.9%的轮次通过率，同时只完成了37.3%的五轮会话；在基于工具的任务上，相邻通过保留率进一步降至52.4%。诊断分析显示，呈现失败集中在信息架构上，交互失败集中在派生状态传播和可供性绑定上，而基于工具的失败还涉及外部状态锚定和需求分解。这些结果将生成式用户界面的评估从评判孤立输出重新定义为测试界面行为、派生状态、外部状态和助手声明在工件演化过程中是否保持同步。

English

Large language models can generate interactive web interfaces, but reliable generative UI requires maintaining an executable artifact as user requests evolve. We introduce EvoGenUI-Bench, a benchmark for multi-turn interface maintenance comprising 150 five-turn tasks and 750 turns across three scenarios: information presentation, executable interaction, and tool-grounded external state. We execute generated artifacts in a browser and evaluate them using screenshots, source and DOM evidence, actor traces, and runtime logs. Beyond turn-level and episode-level success, we measure cross-turn retention with Adjacent Pass Retention. Across eight models, even the strongest achieves 74.9% Turn Pass while completing only 37.3% of five-turn episodes; APR further falls to 52.4% on tool-grounded tasks. Diagnostic analysis shows that presentation failures center on information architecture, interaction failures on derived-state propagation and affordance binding, and tool-grounded failures additionally involve external-state grounding and requirement decomposition. These results reframe generative UI evaluation from judging isolated outputs to testing whether interface behavior, derived state, external state, and assistant claims remain synchronized as the artifact evolves.