从生产流量到后训练：构建覆盖企业请求组合的自托管大语言模型

摘要

数据驻留约束迫使企业自托管大语言模型（LLM），但在不淘汰旧模型的情况下持续引入新模型，会不断扩充服务集群，使本已有限的GPU资源池进一步碎片化。我们通过生产环境错误分析，沿指令遵循、函数调用和内部任务分布三个维度识别并弥合质量差距，将来自200多个内部应用的流量整合到单一模型之上。质量由按生产流量分层构建的离线基准进行追踪，并由确定性验证器或经校准的LLM评估器评分。我们并未采用对所有目标联合优化的方式——这种方法会引入跨域奖励干扰——而是为每个维度分别训练一个GRPO专家，并通过两阶段球面线性插值（SLERP）进行融合。各专家的奖励信号分别暴露一种独特的失败模式：语义坍缩、过度调用和冗长性欺骗，每一种都需要领域针对性的修复方案。在非推理模式下，该方案在内部Arena基准中以69.6对65.8的成绩超越了总参数量约大7倍的基线模型，指令遵循得分为0.85对0.83，函数调用得分为0.79对0.77，同时提升了通用对话基准的表现。该模型以极小比例的服务成本承载了平台50%的流量——即每月1.16亿次请求。

English

Data-residency constraints force enterprises to self-host LLMs, but continuous adoption of newer models without decommissioning their predecessors expands the serving fleet, fragmenting a finite GPU pool. We consolidate traffic from over 200 internal applications onto a single model by closing quality gaps identified through production error analysis along three axes: instruction following, function-calling, and internal task distribution. Quality is tracked by offline benchmarks stratified to production traffic and scored by deterministic verifiers or calibrated LLM judges. Rather than optimising all objectives jointly, which introduces cross-domain reward interference, we train a separate GRPO expert per axis and merge them via two-stage SLERP. Each expert's reward exposes a distinct failure mode, namely semantic collapse, over-calling, and verbosity hacking, each requiring a domain-specific fix. In non-reasoning mode the recipe surpasses a {sim}7times larger by total parameters baseline on the in-house Arena with 69.6 to 65.8, instruction following with 0.85 to 0.83, and function-calling with 0.79 to 0.77, while lifting general dialogue benchmarks. The model absorbs 50% of platform traffic, 116M requests per month, at a fraction of the serving cost.