LLM智能体能够查看代码仓库

摘要

基于大语言模型的编码智能体在软件工程任务中已展现出强劲性能。然而，大多数智能体几乎完全以文本形式处理代码仓库，这与人类开发者利用文件夹层级、依赖关系等视觉结构在大型代码库中定位自身的方式不同。借助多模态大语言模型（MLLMs），智能体能否有效利用代码仓库的视觉表征仍是一个开放性问题。本文首次系统性地实证研究了基于LLM的智能体在仓库级问题解决中应用仓库视觉表征的效果。我们评估了四种最新多模态模型，结果显示：纯视觉模式会降低准确率并增加令牌成本——因为智能体缺乏足够的符号细节，只能通过反复的视觉查询来弥补。与之相反，将仓库结构的可视化图表作为标准文本界面的辅助模态，能更高效地帮助智能体理解结构：输入令牌消耗最多降低26%，同时问题解决准确率得以保持或提升。可视化在故障定位阶段以及智能体自主控制探索深度时最为有用。这些发现为下一代编码智能体提供了实用的文本-视觉混合设计方案。

English

Coding agents powered by large language models have demonstrated strong performance on software engineering tasks. Yet most agents consume repositories almost entirely as text, which differs from how human developers use visual structure such as folder hierarchies and dependency relationships to orient themselves in large codebases. With multimodal large language models (MLLMs), it is an open question whether agents can effectively benefit from visual representations of repositories. This paper presents the first systematic empirical study of visual repository representations for LLM-based agents on repository-level issue resolution. We evaluate four recent multimodal models. Our results show that a strictly vision-only setup degrades accuracy and increases token cost, because agents lack sufficient symbolic detail and compensate with repeated visual queries. In contrast, integrating visual graphs of repository structure as a supplementary modality alongside standard text interfaces helps agents understand structure more efficiently: input token consumption decreases by up to 26% while issue-resolution accuracy is maintained or improved. Visualization is most useful during fault localization and when the agent autonomously controls exploration depth. These findings point to a practical hybrid text-and-vision design for next-generation coding agents.