知识-几何解耦：面向流式推荐的可刷新预训练迁移

摘要

工业推荐系统越来越广泛地采用“预训练-再迁移”范式，然而行为分布漂移带来了两个问题：从行为序列中学习什么，以及在预训练模型持续刷新的情况下如何迁移已学到的知识。为解决这些问题，我们提出了知识-几何解耦（Knowledge-Geometry Decoupling, KGD）。关于“学什么”：传统的下一词元预测将相邻性视为依赖关系，可能编码跨无关会话的虚假转换。我们引入了行为多词元预测（Behavioral Multi-Token Prediction, BMTP），仅保留具有协同或语义关联的未来物品作为监督信号，从而获得更干净、更具可迁移性的行为知识。关于“如何迁移”：预训练知识与任务特定的几何结构对共享参数施加了相互冲突的优化需求。为解决这一问题，KGD将它们分配到独立的参数集中：可刷新的编码器拥有行为知识，而任务学习器通过只读交叉注意力读取上下文编码器状态，并通过与预训练嵌入正交的锚定校准残差（Anchored Calibration Residual, ACR）写入任务特定的几何结构。这种解耦的所有权设计使得知识能够持续刷新，而不会受到任务梯度干扰，也不会使下游适配失效。KGD在八个公开基准上相比强预训练-迁移基线提升了4-12%，并在基线无增益的90天生产数据流上持续保持优势。KGD已在Shopee全面部署。在Shopee首页搜索的在线A/B测试中，它将人均GMV提升了1.75%，广告收入提升了1.53%，展示了其极高的实用价值。我们在https://github.com/FuCongResearchSquad/KGD4REC提供了KGD的核心实现。

English

Industrial recommenders increasingly adopt the pretrain-then-transfer paradigm, yet behavioral distribution drift raises two questions: what to learn from behavior sequences, and how to transfer the learned knowledge while the pretrained model is continually refreshed. To resolve them, we propose Knowledge-Geometry Decoupling (KGD). For what to learn, conventional next-token prediction treats adjacency as dependency and may encode spurious transitions across unrelated sessions. We introduce Behavioral Multi-Token Prediction (BMTP) to retain only collaboratively or semantically related future items as supervision, yielding cleaner and more transferable behavioral knowledge. For how to transfer, pretrained knowledge and task-specific geometry impose conflicting optimization demands on shared parameters. To handle it, KGD assigns them to separate parameter sets: a refreshable encoder owns behavioral knowledge, while a task learner reads contextualized encoder states through read-only cross-attention and writes task-specific geometry through Anchored Calibration Residual (ACR) orthogonal to the pretrained embedding. The decoupled ownership enables continual knowledge refresh without task-gradient interference or invalidating downstream adaptation. KGD improves over strong pretrain-transfer baselines by 4-12% on eight public benchmarks and sustains its advantage over a 90-day production stream where baselines show no gains. KGD has been fully deployed in Shopee. In a live A/B test on Shopee Homepage Search, it increases GMV per user by 1.75% and advertising revenue by 1.53%, demonstrating its high practical value. We provide the core implementation of KGD at https://github.com/FuCongResearchSquad/KGD4REC.