更少的澄清，更好的程式碼：編碼助手中跨會話個人化歧義適應之基準測試

摘要

AI輔助程式設計日益將非正式的使用者意圖轉化為可執行的軟體，然而程式設計請求常包含歧義，這些歧義會在不同任務與工作階段中以使用者特定的方式反覆出現。現有的消歧方法通常僅在當前程式設計工作階段內孤立地處理每個歧義請求，常見方式是徵詢額外的澄清。然而，能否將同一使用者已解決的工作階段歷史作為記憶，用於解決新開啟工作階段中反覆出現的個人化歧義，此問題仍尚未被充分探索。我們將個人化歧義適應定義為一項新任務：給定使用者先前已解決的程式設計工作階段以及一個新的歧義請求，助手應識別反覆出現的歧義模式，產生預期的可執行解決方案，並盡量減少澄清次數。為了對此任務進行基準測試，我們提出了CAPA，它透過六種機制刻畫個人化程式設計歧義，並使用受控的三階段生成管線將這些機制注入無歧義的可執行任務中。CAPA涵蓋60個均衡的使用者-歧義組合，共600個程式設計工作階段，其中包括300個留出的評估工作階段。我們在無歷史紀錄與同一使用者歷史紀錄的條件下，使用可執行成功率、首回合成功率與完成所需回合數來評估12個近期的大型語言模型。我們的分析探討了任務難度、使用者身分與基於記憶的歷史紀錄運用，並進一步提出同使用者歷史門控作為一種輕量級的推論時方法。CAPA為開發長期程式設計助手奠定了基礎，使其能更有效地將生成程式碼與使用者意圖對齊，同時減少重複的澄清需求。

English

AI-assisted coding increasingly translates informal user intent into executable software, yet coding requests often contain ambiguities that recur in user-specific ways across tasks and sessions. Existing disambiguation methods typically address each ambiguous request in isolation within the current coding session, often through eliciting additional clarification. However, whether resolved session history from the same user can serve as memory for resolving recurring personalized ambiguity in a newly opened session remains underexplored. We formulate personalized ambiguity adaptation as a new task: given a user's previously resolved coding sessions and a new ambiguous request, an assistant should identify the recurring ambiguity pattern, produce the intended executable solution, and minimize clarification. To benchmark this task, we introduce CAPA, which characterizes personalized coding ambiguity through six mechanisms and injects these mechanisms into unambiguous executable tasks using a controlled three-stage generation pipeline. CAPA contains 600 coding sessions across 60 balanced user--ambiguity cells, including 300 held-out evaluation sessions. We evaluate 12 recent LLMs under no-history and same-user-history conditions using executable success, first-turn success, and turns-to-completion. Our analyses examine task difficulty, user identity, and memory-based history use, and we further propose same-user history gating as a lightweight inference-time method. CAPA provides a foundation for developing long-term coding assistants that better align generated code with user intent while reducing repeated clarification.