SWE-Touch：当用户修改代码时的编码智能体基准测试

摘要

真实世界的软件开发要求编程智能体在共享工作空间中操作，用户可能在任务进行中检查并修改代码，然而现有的代码库级基准测试通常评估独立工作的智能体，或将用户参与限制为消息交流。这引出了一个问题：编程智能体如何理解并响应共享工作空间中的代码变更？我们提出SWE-Touch，一个通过经校验的反编辑（Counter-Edits）来压力测试这一场景的框架：反编辑是指与任务完成相冲突、但对任务相关代码而言看似合理的编辑。SWE-Touch从多条修复轨迹中挖掘任务关键区域，使用独立的用户补丁生成器来构造编辑，并在智能体到达相关代码时连同上下文相关的用户消息一起注入。我们在SWE-bench Verified上评估了九个编程模型，并在SWE-Bench Pro和DeepSWE的长时程任务上进行了补充实验。反编辑使SWE-bench Verified上的平均解决率降低了7.7个百分点，且这种性能退化在两个长时程基准上同样存在。轨迹分析将这些失败归因于对不断演变的工作空间的认知不足：智能体可能保留冲突代码，或在未充分重新检查代码库、未使用有针对性的测试验证修改后代码的情况下将其替换。这些发现表明，强大的自主性能尚不能确保共享工作空间协作所需的状态感知和自适应行为，并指出检测工作空间变更、协调冲突编辑与任务目标、以及验证受影响行为是未来优化的关键能力。

English

Real-world software development requires coding agents to operate in shared workspaces where users may inspect and modify code during an ongoing task, yet existing repository-level benchmarks typically evaluate agents working alone or restrict user participation to messages. This leads us to ask: how do coding agents understand and respond to code changes in a shared workspace? We introduce SWE-Touch, a framework that stress-tests this setting through validated Counter-Edits: plausible edits to task-relevant code that conflict with task completion. SWE-Touch mines task-critical regions from multiple repair trajectories, uses a separate User Patch Generator to construct the edits, and injects them with contextual user messages when agents reach the relevant code. We evaluate nine coding models on SWE-bench Verified, with additional experiments on longer-horizon tasks from SWE-Bench Pro and DeepSWE. Counter-Edit lowers average resolve rate by 7.7 percentage points on SWE-bench Verified, with degradation also persisting on both longer-horizon benchmarks. Trajectory analysis links these failures to limited awareness of the evolving workspace: agents may retain conflicting code or replace it without sufficiently re-inspecting the repository and validating the revised code with targeted tests. These findings show that strong autonomous performance does not yet ensure the state awareness and adaptive behavior needed for shared-workspace collaboration, and point to detecting workspace changes, reconciling conflicting edits with the task, and verifying the affected behavior as key capabilities for future optimization.