差異與完整檔案：針對 Flutter/Dart 程式碼模型的迭代編輯式與直接生成之實證比較

摘要

大型語言模型用於程式碼編輯時，至少可在兩種輸出模式下訓練與部署：直接生成，模型一次性產生整個修改後的檔案；以及迭代式基於差異（diff）的生成（「步驟」），模型產生一連串局部搜尋/取代編輯，一次套用一個，直到它發出完成訊號或步驟預算耗盡為止。基於 diff 的模式具吸引力，因為它反映了開發者編輯程式碼的方式，且每回合應需要少得多的生成符元。我們在共享的 Flutter/Dart 資料集上，以這兩種模式訓練兩個程式碼模型——一個從頭訓練的 1 億參數模型（Rainbow-Pony-100M），以及一個微調的 Qwen2.5-Coder-0.5B——並在每個模型約 1,790 個任務的留出集上，評估所有四個所得模型。直接生成在我們測量的每一項指標上都大幅優於基於差異的生成——編譯/靜態分析通過率、每位元組位元數（bits-per-byte）、與參考答案的字元層級相似度，以及盲式 LLM 評審對目標達成、正確性與程式碼品質的評分——且此差距在透過匹配 ID 比較控制任務難度後，以及僅限於雙方皆可編譯的程式碼時，仍然存在。接著，我們找出一個單一且與架構無關的機制，解釋基於差異的生成確實在哪些條件下勝出：它在短小、空間上局部的編輯上具有競爭力，且其類別層級的勝出正好集中在我們資料集中平均編輯步驟數最低的兩個任務類別——重構，以及錯誤處理/邊界情況修正。我們將此稱為任務局部性（task locality），並討論其對於以編輯為基礎的訓練模式何時是、何時不是程式碼編輯模型正確選擇的意涵。

English

Large language models used for code editing can be trained and deployed in at least two output regimes: direct generation, where the model emits the entire modified file in one shot, and iterative diff-based generation ("steps"), where the model emits a sequence of localized search/replace edits applied one at a time until it signals completion or a step budget is exhausted. The diff-based regime is attractive because it mirrors how developers edit code and should require far fewer generated tokens per turn. We train two code models - a 100M-parameter model trained from scratch (Rainbow-Pony-100M) and a fine-tuned Qwen2.5-Coder-0.5B - in both regimes on a shared Flutter/Dart dataset, and evaluate all four resulting models on a held-out set of approx 1,790 tasks per model. Direct generation substantially outperforms diff-based generation on every metric we measure - compilation/static-analysis pass rate, bits-per-byte, character-level similarity to the reference, and blinded LLM-judge ratings of goal fulfillment, correctness, and code quality - and the gap persists after controlling for task difficulty via a matched-ID comparison and when restricting to code that compiles on both sides. We then identify a single, architecture-independent mechanism behind the conditions where diff-based generation does win: it is competitive on short, spatially localized edits, and its category-level wins concentrate in exactly the two task categories - refactoring and error-handling/edge-case fixes - with the lowest mean edit-step count in our dataset. We term this task locality and discuss its implications for when an edit-based training regime is and is not the right choice for a code-editing model.