使用者LLM：具有使用者嵌入的高效LLM情境化

摘要

大型語言模型（LLMs）已經徹底改變了自然語言處理。然而，有效地整合複雜且可能帶有噪音的用戶互動數據仍然是一個挑戰。為了應對這一問題，我們提出了User-LLM，這是一個新穎的框架，利用用戶嵌入來將LLMs置於上下文中。這些嵌入是通過自監督預訓練從各種用戶互動中提煉出來的，捕捉了潛在的用戶偏好以及隨時間演變的情況。我們通過交叉注意力和軟提示將這些用戶嵌入與LLMs集成，使LLMs能夠動態適應用戶上下文。我們在MovieLens、Amazon Review和Google Local Review數據集上進行了全面的實驗，展示了在各種任務上顯著的性能提升。值得注意的是，我們的方法在長序列任務和需要深入了解用戶的任務上優於基於文本提示的情境化方法，同時保持了計算效率。我們進一步將Perceiver層整合到用戶編碼器和LLMs之間，以簡化集成，降低計算需求。

English

Large language models (LLMs) have revolutionized natural language processing. However, effectively incorporating complex and potentially noisy user interaction data remains a challenge. To address this, we propose User-LLM, a novel framework that leverages user embeddings to contextualize LLMs. These embeddings, distilled from diverse user interactions using self-supervised pretraining, capture latent user preferences and their evolution over time. We integrate these user embeddings with LLMs through cross-attention and soft-prompting, enabling LLMs to dynamically adapt to user context. Our comprehensive experiments on MovieLens, Amazon Review, and Google Local Review datasets demonstrate significant performance gains across various tasks. Notably, our approach outperforms text-prompt-based contextualization on long sequence tasks and tasks that require deep user understanding while being computationally efficient. We further incorporate Perceiver layers to streamline the integration between user encoders and LLMs, reducing computational demands.

使用者LLM：具有使用者嵌入的高效LLM情境化

User-LLM: Efficient LLM Contextualization with User Embeddings

摘要

Support