RepoRescue：关于LLM智能体进行全仓库兼容性修复的实证研究

摘要

开源库和工具被广泛复用，但兼容性维护成本高昂。一旦维护者离开，随着运行环境和依赖项的演进，原本有用的仓库可能会停止工作。我们研究LLM智能体是否能使旧仓库适应现代环境，我们将此任务称为兼容性救援（compatibility rescue）。与漏洞修复不同，兼容性救援的起点是一个在原始环境中能正常工作、但因生态漂移而失效的仓库。RepoRescue仅向智能体提供该仓库及其失效的现代环境；智能体必须诊断故障原因、定位受影响的代码，并生成能恢复历史测试套件的源码级救援方案。我们从193个Python仓库和122个Java仓库构建了RepoRescue，每个仓库均经验证：在历史环境中测试通过，而在现代化后测试失败。我们在Python上评估了五个已部署的智能体系统，在Java上评估了三个。除完整补丁通过率外，我们还通过移除测试文件编辑后重新运行补丁来评估仅源码修复的效果；增加一个运行时强制机制以阻止测试文件编辑；并验证那些在救援后测试套件通过的仓库的实际可用性。我们发现，即使被提示不要编辑，Claude Code系统有时仍会修改失败的测试；但在运行时阻塞机制下，Kimi仍能成功救援41.5%的仓库。各系统具有互补性：它们的并集救援率达到62.7%，超过最佳单一系统10.9个百分点。难点集中于跨文件协调：在14个需要全代码库协调修改的仓库中，GPT-5.2通过Codex实现全部通过，而每个Claude Code系统至多通过两个。最后，测试套件通过仅是初步信号：在34个未维护的Python候选仓库（其套件在救援后通过）中，有22个在真实场景中可正常工作，12个通过了补丁修复兼容性故障的漏洞检查。RepoRescue通过仅源码审计、运行时强制、实际验证和推理标签，对兼容性救援进行基准测试。

English

Open-source libraries and tools are widely reused, but compatibility maintenance is expensive. Once maintainers leave, useful repositories can stop working as runtimes and dependencies evolve. We study whether LLM agents can adapt old repositories to modern environments, a task we call compatibility rescue. Unlike bug repair, compatibility rescue starts from a repository that worked in its original environment but fails after ecosystem drift. RepoRescue gives agents only the repository and its failing modern environment; the agent must diagnose the failure, locate affected code, and produce a source-code rescue that restores the historical test suite. We build RepoRescue from 193 Python and 122 Java repositories, each verified to pass historically and fail after modernization. We evaluate five deployed agent systems on Python and three on Java. Beyond full-patch pass rate, we rerun patches after removing test-file edits to measure source-only repair, add a runtime-enforced regime that blocks test edits, and validate practical use for repositories whose suites pass after rescue. We find that Claude Code systems sometimes edit failing tests even when prompted not to; with runtime blocking, Kimi still rescues 41.5% of repositories. Systems are complementary: their union reaches 62.7%, exceeding the best single system by 10.9 points. Difficulty concentrates in cross-file coordination: on 14 repositories requiring coordinated whole-codebase changes, GPT-5.2 through Codex passes all 14, while every Claude Code system passes at most two. Finally, a passing suite is only an initial signal: among 34 unmaintained Python candidates whose suites pass after rescue, 22 work in realistic scenarios and 12 pass bug-hunt with patches that address the compatibility failure. RepoRescue benchmarks compatibility rescue with source-only auditing, runtime enforcement, practical validation, and reasoning labels.