X-Cross：跨域序列推荐中语言模型的动态集成

X-Cross: Dynamic Integration of Language Models for Cross-Domain Sequential Recommendation

April 29, 2025

作者: Guy Hadad, Haggai Roitman, Yotam Eshel, Bracha Shapira, Lior Rokach

cs.AI

摘要

随着新产品日新月异，推荐系统需快速适应可能的新领域，而无需大量重新训练。本研究提出“X-Cross”——一种新颖的跨域序列推荐模型，通过整合多个领域特定的语言模型来推荐新领域的产品；每个模型均采用低秩适配器（LoRA）进行微调。面对推荐提示，X-Cross逐层操作，动态精炼每个源语言模型的表示，通过融合所有其他模型的知识实现。这些精炼后的表示从一层传递至下一层，利用各领域适配器的激活，确保在保持领域特定细微差异的同时，实现跨域适应性。利用亚马逊数据集进行序列推荐，X-Cross在仅使用25%额外参数的情况下，达到了与LoRA微调模型相当的性能。在跨域任务中，如从玩具领域适应至工具、电子或体育领域，X-Cross展现了强劲性能，且相比LoRA，所需微调数据量减少约50%-75%，使微调更为高效。此外，X-Cross在准确性上显著优于其他跨域基线方法。总体而言，X-Cross实现了可扩展且自适应的跨域推荐，降低了计算开销，为数据受限环境提供了高效解决方案。

English

As new products are emerging daily, recommendation systems are required to quickly adapt to possible new domains without needing extensive retraining. This work presents ``X-Cross'' -- a novel cross-domain sequential-recommendation model that recommends products in new domains by integrating several domain-specific language models; each model is fine-tuned with low-rank adapters (LoRA). Given a recommendation prompt, operating layer by layer, X-Cross dynamically refines the representation of each source language model by integrating knowledge from all other models. These refined representations are propagated from one layer to the next, leveraging the activations from each domain adapter to ensure domain-specific nuances are preserved while enabling adaptability across domains. Using Amazon datasets for sequential recommendation, X-Cross achieves performance comparable to a model that is fine-tuned with LoRA, while using only 25% of the additional parameters. In cross-domain tasks, such as adapting from Toys domain to Tools, Electronics or Sports, X-Cross demonstrates robust performance, while requiring about 50%-75% less fine-tuning data than LoRA to make fine-tuning effective. Furthermore, X-Cross achieves significant improvement in accuracy over alternative cross-domain baselines. Overall, X-Cross enables scalable and adaptive cross-domain recommendations, reducing computational overhead and providing an efficient solution for data-constrained environments.