GitHub仓库中AI使用特征与演化的实证研究：来自代码注释的证据

摘要

开发者在日常软件工作流中越来越多地使用诸如ChatGPT、Copilot和Claude等AI工具，但以往的研究往往孤立评估大语言模型的输出，而非考察开发者在实际项目中如何调整这些输出。我们分析了35,361条明确提及AI使用的GitHub代码注释及其关联代码块。首先对500条独特注释及代码块进行开放式编码，构建AI辅助开发活动的分类体系；随后使用两个基于大语言模型的分类器对完整数据集进行标注，并采用Dawid-Skene期望最大化方法聚合预测结果。此外，我们分析了12,996条后续提交信息，探究AI辅助代码在引入后的演变过程，并考察了2022年12月至2026年3月期间的时间趋势。研究结果表明，开发者主要将大语言模型用于代码实现，其次是代码增强、调试、文档编写和测试。后续提交频繁涉及重构与清理、功能集成与扩展以及缺陷修复，表明开发者在适配AI辅助代码时持续进行人工监督。随时间推移，引用AI的注释从直接代码生成转向知识与概念支持以及代码增强。这些发现表明，AI工具不仅作为代码生成辅助手段嵌入开发流程，更成为协作支持机制——其输出结果由开发者持续进行精炼、扩展与修正。

English

Developers increasingly use AI tools such as ChatGPT, Copilot, and Claude in everyday software workflows, but prior studies often evaluate LLM outputs in isolation rather than examining how developers adapt them in real projects. We analyze 35,361 GitHub code comments that explicitly reference AI use and their associated code blocks. We first open-code 500 unique comments and code blocks to derive a taxonomy of AI-assisted development activities, then annotate the full dataset using two LLM-based classifiers and aggregate predictions with Dawid-Skene expectation-maximization. We also analyze 12,996 subsequent commit messages to study how AI-assisted code evolves after introduction, and examine temporal trends from December 2022 to March 2026. Our results show that developers primarily use LLMs for code implementation, followed by code enhancement, debugging, documentation, and testing. Subsequent commits frequently involve refactoring and cleanup, feature integration and extension, and bug fixing, indicating sustained human oversight in adapting AI-assisted code. Over time, AI-referencing comments shift from direct code generation toward knowledge and conceptual support and code enhancement. These findings suggest that AI tools are becoming embedded not only as code-generation aids, but also as collaborative support mechanisms whose outputs are refined, extended, and corrected by developers over time.