EvoGenUI-Bench：評估大型語言模型作為多輪生成式使用者介面助理

摘要

大型語言模型能生成互動式網頁介面，但可靠的生成式UI需要在使用者需求演變的過程中，持續維護一個可執行的成品。我們提出EvoGenUI-Bench，這是一個用於多輪介面維護的評測基準，包含150個五輪任務、共750個回合，涵蓋三種情境：資訊呈現、可執行互動，以及工具對接的外部狀態。我們在瀏覽器中執行生成的成品，並利用螢幕截圖、原始碼與DOM證據、操作者軌跡以及執行時日誌進行評估。除了回合層級與任務層級的成功率之外，我們也以「相鄰通過留存率」(Adjacent Pass Retention, APR) 來衡量跨回合的留存能力。在八個模型中，即便最強的模型，其回合通過率僅達74.9%，而五輪任務的完成率僅有37.3%；在工具對接的任務上，APR更進一步降至52.4%。診斷分析顯示，呈現類失敗主要集中於資訊架構；互動類失敗則在於衍生狀態傳播與可供性綁定；而工具對接類失敗還涉及外部狀態的對接與需求分解。這些結果將生成式UI的評估，從評判孤立的輸出，轉向測試介面行為、衍生狀態、外部狀態以及助理聲明之間，在成品演進過程中是否保持同步。

English

Large language models can generate interactive web interfaces, but reliable generative UI requires maintaining an executable artifact as user requests evolve. We introduce EvoGenUI-Bench, a benchmark for multi-turn interface maintenance comprising 150 five-turn tasks and 750 turns across three scenarios: information presentation, executable interaction, and tool-grounded external state. We execute generated artifacts in a browser and evaluate them using screenshots, source and DOM evidence, actor traces, and runtime logs. Beyond turn-level and episode-level success, we measure cross-turn retention with Adjacent Pass Retention. Across eight models, even the strongest achieves 74.9% Turn Pass while completing only 37.3% of five-turn episodes; APR further falls to 52.4% on tool-grounded tasks. Diagnostic analysis shows that presentation failures center on information architecture, interaction failures on derived-state propagation and affordance binding, and tool-grounded failures additionally involve external-state grounding and requirement decomposition. These results reframe generative UI evaluation from judging isolated outputs to testing whether interface behavior, derived state, external state, and assistant claims remain synchronized as the artifact evolves.