VisualLens：通过视觉历史个性化

VisualLens: Personalization through Visual History

November 25, 2024

作者: Wang Bill Zhu, Deqing Fu, Kai Sun, Yi Lu, Zhaojiang Lin, Seungwhan Moon, Kanika Narang, Mustafa Canim, Yue Liu, Anuj Kumar, Xin Luna Dong

cs.AI

摘要

我们假设用户的视觉历史，即反映其日常生活的图像，提供了有价值的洞见，能够揭示他们的兴趣和偏好，并可用于个性化定制。在实现这一目标时，面临诸多挑战中，首要挑战是视觉历史中的多样性和噪音，其中包含不一定与推荐任务相关、不一定反映用户兴趣，甚至不一定与偏好相关的图像。现有的推荐系统要么依赖于特定任务的用户交互日志，例如在线购物历史用于购物推荐，要么侧重于文本信号。我们提出了一种新颖的方法，VisualLens，用于提取、过滤和优化图像表示，并利用这些信号进行个性化定制。我们创建了两个新的基准测试，具有任务无关的视觉历史，并展示了我们的方法在Hit@3上比最先进的推荐提高了5-10%，在GPT-4o上提高了2-5%。我们的方法为在传统方法失败的情景下的个性化推荐铺平了道路。

English

We hypothesize that a user's visual history with images reflecting their daily life, offers valuable insights into their interests and preferences, and can be leveraged for personalization. Among the many challenges to achieve this goal, the foremost is the diversity and noises in the visual history, containing images not necessarily related to a recommendation task, not necessarily reflecting the user's interest, or even not necessarily preference-relevant. Existing recommendation systems either rely on task-specific user interaction logs, such as online shopping history for shopping recommendations, or focus on text signals. We propose a novel approach, VisualLens, that extracts, filters, and refines image representations, and leverages these signals for personalization. We created two new benchmarks with task-agnostic visual histories, and show that our method improves over state-of-the-art recommendations by 5-10% on Hit@3, and improves over GPT-4o by 2-5%. Our approach paves the way for personalized recommendations in scenarios where traditional methods fail.