X-Cross：跨領域序列推薦中語言模型的動態整合

X-Cross: Dynamic Integration of Language Models for Cross-Domain Sequential Recommendation

April 29, 2025

作者: Guy Hadad, Haggai Roitman, Yotam Eshel, Bracha Shapira, Lior Rokach

cs.AI

摘要

隨著新產品日新月異地湧現，推薦系統需要能夠快速適應可能出現的新領域，而無需進行大量的重新訓練。本研究提出了「X-Cross」——一種新穎的跨領域序列推薦模型，該模型通過整合多個領域特定的語言模型來推薦新領域中的產品；每個模型均通過低秩適配器（LoRA）進行微調。面對推薦提示時，X-Cross逐層操作，動態地精煉每個源語言模型的表示，通過整合來自所有其他模型的知識來實現。這些精煉後的表示從一層傳播到下一層，利用每個領域適配器的激活，確保在保持領域特定細微差別的同時，實現跨領域的適應性。使用亞馬遜數據集進行序列推薦時，X-Cross的表現與使用LoRA微調的模型相當，而僅使用了25%的額外參數。在跨領域任務中，例如從玩具領域適應到工具、電子產品或體育領域，X-Cross展現了強健的性能，同時相比LoRA，需要約50%-75%更少的微調數據來使微調有效。此外，X-Cross在準確性上相較於其他跨領域基線模型取得了顯著提升。總體而言，X-Cross實現了可擴展且自適應的跨領域推薦，降低了計算開銷，為數據受限的環境提供了一個高效的解決方案。

English

As new products are emerging daily, recommendation systems are required to quickly adapt to possible new domains without needing extensive retraining. This work presents ``X-Cross'' -- a novel cross-domain sequential-recommendation model that recommends products in new domains by integrating several domain-specific language models; each model is fine-tuned with low-rank adapters (LoRA). Given a recommendation prompt, operating layer by layer, X-Cross dynamically refines the representation of each source language model by integrating knowledge from all other models. These refined representations are propagated from one layer to the next, leveraging the activations from each domain adapter to ensure domain-specific nuances are preserved while enabling adaptability across domains. Using Amazon datasets for sequential recommendation, X-Cross achieves performance comparable to a model that is fine-tuned with LoRA, while using only 25% of the additional parameters. In cross-domain tasks, such as adapting from Toys domain to Tools, Electronics or Sports, X-Cross demonstrates robust performance, while requiring about 50%-75% less fine-tuning data than LoRA to make fine-tuning effective. Furthermore, X-Cross achieves significant improvement in accuracy over alternative cross-domain baselines. Overall, X-Cross enables scalable and adaptive cross-domain recommendations, reducing computational overhead and providing an efficient solution for data-constrained environments.