CodexGraph：通過程式碼圖資料庫搭建大型語言模型與程式碼存儲庫之間的橋樑

CodexGraph: Bridging Large Language Models and Code Repositories via Code Graph Databases

August 7, 2024

作者: Xiangyan Liu, Bo Lan, Zhiyuan Hu, Yang Liu, Zhicheng Zhang, Wenmeng Zhou, Fei Wang, Michael Shieh

cs.AI

摘要

大型語言模型（LLMs）在像HumanEval和MBPP這樣的獨立代碼任務中表現出色，但在處理整個代碼庫時卻遇到困難。這一挑戰促使研究人員開展研究，以增強LLM-代碼庫之間的互動，實現代碼庫規模上的提升。目前的解決方案依賴基於相似性的檢索或手動工具和API，但各自存在顯著缺點。基於相似性的檢索在複雜任務中通常召回率低，而手動工具和API通常是特定任務的，需要專業知識，降低了它們在各種代碼任務和現實應用中的通用性。為了克服這些限制，我們引入了CodexGraph，這是一個系統，將LLM代理與從代碼庫中提取的圖形數據庫接口集成在一起。通過利用圖形數據庫的結構特性和圖形查詢語言的靈活性，CodexGraph使LLM代理能夠構建和執行查詢，從而實現精確、代碼結構感知的上下文檢索和代碼導航。我們使用三個基準測試CrossCodeEval、SWE-bench和EvoCodeBench來評估CodexGraph。此外，我們開發了五個現實世界的編碼應用程序。通過統一的圖形數據庫模式，CodexGraph展示了在學術和現實環境中的競爭性表現和潛力，展示了它在軟件工程中的多功能性和有效性。我們的應用演示： https://github.com/modelscope/modelscope-agent/tree/master/apps/codexgraph_agent.

English

Large Language Models (LLMs) excel in stand-alone code tasks like HumanEval and MBPP, but struggle with handling entire code repositories. This challenge has prompted research on enhancing LLM-codebase interaction at a repository scale. Current solutions rely on similarity-based retrieval or manual tools and APIs, each with notable drawbacks. Similarity-based retrieval often has low recall in complex tasks, while manual tools and APIs are typically task-specific and require expert knowledge, reducing their generalizability across diverse code tasks and real-world applications. To mitigate these limitations, we introduce \framework, a system that integrates LLM agents with graph database interfaces extracted from code repositories. By leveraging the structural properties of graph databases and the flexibility of the graph query language, \framework enables the LLM agent to construct and execute queries, allowing for precise, code structure-aware context retrieval and code navigation. We assess \framework using three benchmarks: CrossCodeEval, SWE-bench, and EvoCodeBench. Additionally, we develop five real-world coding applications. With a unified graph database schema, \framework demonstrates competitive performance and potential in both academic and real-world environments, showcasing its versatility and efficacy in software engineering. Our application demo: https://github.com/modelscope/modelscope-agent/tree/master/apps/codexgraph_agent.