從生產流量到後訓練：建構涵蓋企業請求組合的自架LLM

摘要

資料駐留限制迫使企業自行託管大型語言模型，但持續採用新模型而不淘汰舊模型，會導致服務叢集不斷擴張，使有限的 GPU 資源更加碎片化。我們透過生產環境錯誤分析，沿三個軸向彌合品質差距，將來自 200 多個內部應用程式的流量整合到單一模型上：指令遵循、函式呼叫及內部任務分佈。品質透過依生產流量分層的離線基準來追蹤，並由確定性驗證器或經校準的 LLM 評審員評分。我們不採用聯合優化所有目標的做法（此舉會引入跨領域獎勵干擾），而是針對每個軸向分別訓練一個 GRPO 專家模型，並透過兩階段 SLERP 進行合併。每個專家的獎勵函數揭示了不同的失敗模式，分別為語義崩潰、過度呼叫及冗長駭入，每一種都需要針對特定領域的修正。在非推理模式下，此方法在內部 Arena 上超越了總參數量約 7 倍的基準模型，指標分別為 69.6 對 65.8；指令遵循為 0.85 對 0.83；函式呼叫為 0.79 對 0.77，同時提升了通用對話基準表現。該模型以極低的服務成本吸收了平台 50% 的流量，即每月 1.16 億次請求。

English

Data-residency constraints force enterprises to self-host LLMs, but continuous adoption of newer models without decommissioning their predecessors expands the serving fleet, fragmenting a finite GPU pool. We consolidate traffic from over 200 internal applications onto a single model by closing quality gaps identified through production error analysis along three axes: instruction following, function-calling, and internal task distribution. Quality is tracked by offline benchmarks stratified to production traffic and scored by deterministic verifiers or calibrated LLM judges. Rather than optimising all objectives jointly, which introduces cross-domain reward interference, we train a separate GRPO expert per axis and merge them via two-stage SLERP. Each expert's reward exposes a distinct failure mode, namely semantic collapse, over-calling, and verbosity hacking, each requiring a domain-specific fix. In non-reasoning mode the recipe surpasses a {sim}7times larger by total parameters baseline on the in-house Arena with 69.6 to 65.8, instruction following with 0.85 to 0.83, and function-calling with 0.79 to 0.77, while lifting general dialogue benchmarks. The model absorbs 50% of platform traffic, 116M requests per month, at a fraction of the serving cost.