我們在實際功能級程式修復方面能走多遠？

How Far Can We Go with Practical Function-Level Program Repair?

April 19, 2024

作者: Jiahong Xiang, Xiaoyang Xu, Fanchu Kong, Mingyuan Wu, Haotian Zhang, Yuqun Zhang

cs.AI

摘要

最近，基於大型語言模型（LLMs）的多種自動程序修復（APR）技術已被提出，以增強修復性能。儘管這些技術主要集中在單行或塊級別的修復，但由於修復任務範圍有限且昂貴的語句級錯誤定位，它們在實際應用中面臨著重大挑戰。然而，更實用的功能級APR將修復任務範圍擴展到修復整個有錯誤的功能，並僅需要成本效益高的功能級錯誤定位，但這方面的研究仍未深入探討。本文首次對基於LLM的功能級APR進行了全面研究，包括調查少樣本學習機制和輔助修復相關信息的影響。具體來說，我們採用了六種廣泛研究的LLMs，在Defects4J 1.2和2.0數據集中構建了一個基準。我們的研究表明，具有零樣本學習的LLMs已經是功能級APR技術的強大工具，而應用少樣本學習機制則導致不同的修復性能。此外，我們發現將輔助修復相關信息直接應用於LLMs顯著提高了功能級修復性能。受到我們研究結果的啟發，我們提出了一種基於LLM的功能級APR技術，名為SRepair，該技術採用雙LLM框架，以利用輔助修復相關信息的強大功能來提高修復性能。評估結果表明，SRepair可以在Defects4J數據集中正確修復300個單功能錯誤，至少比所有先前的APR技術高出85％，而無需昂貴的語句級錯誤定位信息。此外，SRepair成功修復了Defects4J數據集中的32個多功能錯誤，這是我們所知道的任何APR技術首次實現。

English

Recently, multiple Automated Program Repair (APR) techniques based on Large Language Models (LLMs) have been proposed to enhance the repair performance. While these techniques mainly focus on the single-line or hunk-level repair, they face significant challenges in real-world application due to the limited repair task scope and costly statement-level fault localization. However, the more practical function-level APR, which broadens the scope of APR task to fix entire buggy functions and requires only cost-efficient function-level fault localization, remains underexplored. In this paper, we conduct the first comprehensive study of LLM-based function-level APR including investigating the effect of the few-shot learning mechanism and the auxiliary repair-relevant information. Specifically, we adopt six widely-studied LLMs and construct a benchmark in both the Defects4J 1.2 and 2.0 datasets. Our study demonstrates that LLMs with zero-shot learning are already powerful function-level APR techniques, while applying the few-shot learning mechanism leads to disparate repair performance. Moreover, we find that directly applying the auxiliary repair-relevant information to LLMs significantly increases function-level repair performance. Inspired by our findings, we propose an LLM-based function-level APR technique, namely SRepair, which adopts a dual-LLM framework to leverage the power of the auxiliary repair-relevant information for advancing the repair performance. The evaluation results demonstrate that SRepair can correctly fix 300 single-function bugs in the Defects4J dataset, largely surpassing all previous APR techniques by at least 85%, without the need for the costly statement-level fault location information. Furthermore, SRepair successfully fixes 32 multi-function bugs in the Defects4J dataset, which is the first time achieved by any APR technique ever to our best knowledge.