RepoRescue：LLM智能體在整個倉庫兼容性救援的實證研究

摘要

開源函式庫與工具被廣泛重用，但相容性維護成本高昂。一旦維護者離開，隨著執行環境與依賴項的演進，原先可用的儲存庫可能停止運作。我們研究大型語言模型代理是否能將老舊儲存庫調整至現代環境——這項任務稱為「相容性救援」。與錯誤修復不同，相容性救援始於一個在原環境中可正常運作、但因生態系統漂移而失效的儲存庫。RepoRescue 僅提供代理該儲存庫及其失效的現代環境；代理必須診斷失敗原因、定位受影響的程式碼，並產出能恢復歷史測試套件的原始碼救援方案。我們基於 193 個 Python 儲存庫與 122 個 Java 儲存庫建構 RepoRescue，每個儲存庫皆經驗證在歷史環境中通過測試，而在現代化後則失敗。我們評估了五個部署於 Python 的代理系統及三個部署於 Java 的系統。除完整修補的通過率外，我們在移除測試檔案編輯後重新執行修補，以衡量「純原始碼修復」；加入執行時強制機制，阻止測試編輯；並驗證儲存庫在救援後測試套件通過的實際可用性。我們發現，即使提示不要修改，Claude Code 系統有時仍會編輯失敗的測試；在執行時封鎖下，Kimi 仍能救援 41.5% 的儲存庫。各系統具有互補性：其聯集達 62.7%，超過最佳單一系統 10.9 個百分點。困難集中於跨檔案協調：在 14 個需要全程式碼庫協調變更的儲存庫中，GPT-5.2 透過 Codex 全部通過，而每個 Claude Code 系統最多僅通過兩個。最後，通過測試套件僅是初步訊號：在 34 個未維護、救援後測試套件通過的 Python 候選儲存庫中，22 個在實際情境中運作正常，12 個通過錯誤狩獵，且修補能確實解決相容性失敗。RepoRescue 以純原始碼審查、執行時強制、實務驗證與推理標籤來評測相容性救援。

English

Open-source libraries and tools are widely reused, but compatibility maintenance is expensive. Once maintainers leave, useful repositories can stop working as runtimes and dependencies evolve. We study whether LLM agents can adapt old repositories to modern environments, a task we call compatibility rescue. Unlike bug repair, compatibility rescue starts from a repository that worked in its original environment but fails after ecosystem drift. RepoRescue gives agents only the repository and its failing modern environment; the agent must diagnose the failure, locate affected code, and produce a source-code rescue that restores the historical test suite. We build RepoRescue from 193 Python and 122 Java repositories, each verified to pass historically and fail after modernization. We evaluate five deployed agent systems on Python and three on Java. Beyond full-patch pass rate, we rerun patches after removing test-file edits to measure source-only repair, add a runtime-enforced regime that blocks test edits, and validate practical use for repositories whose suites pass after rescue. We find that Claude Code systems sometimes edit failing tests even when prompted not to; with runtime blocking, Kimi still rescues 41.5% of repositories. Systems are complementary: their union reaches 62.7%, exceeding the best single system by 10.9 points. Difficulty concentrates in cross-file coordination: on 14 repositories requiring coordinated whole-codebase changes, GPT-5.2 through Codex passes all 14, while every Claude Code system passes at most two. Finally, a passing suite is only an initial signal: among 34 unmaintained Python candidates whose suites pass after rescue, 22 work in realistic scenarios and 12 pass bug-hunt with patches that address the compatibility failure. RepoRescue benchmarks compatibility rescue with source-only auditing, runtime enforcement, practical validation, and reasoning labels.