強化學習中從評分量表到程式碼的信用分配

摘要

互動式網頁應用程式生成要求模型能根據自然語言請求，產出可用的 HTML、CSS 與 JavaScript 應用程式。與傳統程式碼生成不同，應用程式品質取決於多個面向使用者的功能性需求，而每項需求通常對應到局部化的程式碼區域，例如事件處理器、狀態更新、DOM 片段或 CSS 選擇器。標準 GRPO 將這些結構化結果壓縮為單一的序列層級獎勵，並將所得優勢均勻地應用於所有 token，從而削弱了信用分配的效果。我們提出 Rubric-to-Code Credit Assignment（RCCA），這是一個強化學習框架，能將評分標準層級的功能性回饋轉化為生成程式碼上的局部最佳化訊號。RCCA 圍繞明確的功能性評分標準建構訓練任務，使用分層獎勵來區分格式、原始碼、執行時期與功能性失敗，並將評估器生成的文本歸因與對應的程式碼區段及生成 token 對齊。由此產生的模型 Ling-RCCA-Flash 在 MiniAppBench 上取得 41.25 分，較 Ling-3.0-Flash 提升 32.20 分，並小幅超越 Claude Opus 4.5。該模型在 ArtifactsBench 上亦達到 76.19 分，較 SFT 模型提升 4.48 分，並在官方 ArtifactsBench 排行榜設定下以超越 GPT-5 3.64 分的成績創下新高，顯示出可遷移的實作層級增益。

English

Interactive web application generation requires models to produce usable HTML, CSS, and JavaScript applications from natural language requests. Unlike conventional code generation, application quality depends on multiple user-facing functional requirements, each often tied to localized code regions such as event handlers, state updates, DOM fragments, or CSS selectors. Standard GRPO collapses these structured outcomes into a single sequence-level reward and applies the resulting advantage uniformly to all tokens, weakening credit assignment. We propose Rubric-to-Code Credit Assignment (RCCA), a reinforcement learning framework that converts rubric-level functional feedback into localized optimization signals over generated code. RCCA builds training tasks around explicit functional rubrics, uses a hierarchical reward to separate format, source-code, runtime, and functional failures, and aligns evaluator-generated textual attributions with responsible code spans and generated tokens. The resulting model, Ling-RCCA-Flash, scores 41.25 on MiniAppBench, improving Ling-3.0-Flash by 32.20 points and slightly surpassing Claude Opus 4.5. It also reaches 76.19 on ArtifactsBench, improving the SFT model by 4.48 points and establishing a new top score under the official ArtifactsBench leaderboard setting by surpassing the GPT-5 score by 3.64 points, suggesting transferable implementation-level gains.