教会大型语言模型低资源语言：提升Pharo中的代码补全

摘要

大型语言模型（LLMs）为自动化代码编写开辟了新的可能性，成为大多数代码补全工具的支柱。尽管LLMs在主流语言中表现出色，但它们往往缺乏对所谓低资源语言的支持——这些语言的训练数据稀缺。因此，这些语言在代码补全工具的质量上落后于其社区所能获得的水平。一个具体例子是Pharo，它是一种受Smalltalk启发的语言，其集成开发环境（IDE）目前仅提供单标记补全。在本研究中，我们汇报了将基于LLM的代码补全引入Pharo的经验。首先，我们描述了一个端到端流水线，该流水线结合了Pharo特有的数据整理、对开源代码LLMs的持续预训练和微调。其次，我们介绍了一套Pharo代码补全基准测试，旨在评估模型是否（i）学习Pharo的语法，以及（ii）准确补全来自真实GitHub仓库的掩码Pharo代码。第三，我们通过实验证明，专用于Pharo的模型在性能上大幅超越其原始基础检查点，并且在Pharo补全任务上也超过了规模更大的代码LLMs的准确性。总体而言，我们的案例研究证明了将强大的基于LLM的代码补全引入低资源编程语言的可行性，这些模型的规模足够小，可在IDE中提供“实时”支持。

English

Large Language Models (LLMs) unlocked new possibilities in automated code writing, becoming the backbone of most code completion tools. While LLMs excel in mainstream languages, they often lack support for the so-called low-resource languages where training data is scarce. As a result, these languages lag behind in the quality of code completion tooling available to their communities. A concrete example is Pharo, a Smalltalk-inspired language whose IDE currently offers only single-token completion. In this work, we report on our experience bringing LLM-based code completion to Pharo. First, we describe an end-to-end pipeline that combines Pharo-specific data curation, continued pre-training and fine-tuning of open code LLMs. Second, we introduce a set of Pharo code completion benchmarks designed to evaluate whether models (i) learn Pharo's syntax and (ii) accurately complete masked Pharo code from real-world GitHub repositories. Third, we show empirically that Pharo-specialized models substantially outperform their original base checkpoints and also exceed the accuracy of substantially larger code LLMs on Pharo completion. Overall, our case study demonstrates the feasibility of bringing strong LLM-based code completion to low-resource programming languages, with models small enough to provide ``real-time'' in-IDE support.