一个污染页面足矣：评估LLM推荐系统中的网络内容污染

摘要

搜索增强的大语言模型通过检索实时网络内容，日益成为日常消费推荐的中介。这带来了一种新风险：大语言模型推荐器可能会消费被生成式引擎优化（GEO）运营者污染的网络内容，从而被误导。我们提出以下问题：它们在多大程度上会沦为虚假产品的无意推广者？我们引入FORGE（生成式环境中的虚假在线推荐）基准，该基准在固定的检索网页集合中，将真实产品局部改写为虚假产品，并衡量大语言模型推荐虚假产品的频率——涵盖15个类别、5种消费场景中的225个真实产品。在12个商业及开源权重的大语言模型中，所有模型均存在脆弱性：单一污染页面即可造成高达27%的受骗率，而完整的top-3替换可将这一比例提升至73.8%。脆弱性因类别而异，当模型缺乏对产品的稳定先验知识时，脆弱性会加剧。推理并不能缓解这一脆弱性；相反，它往往生成虚假的社会认同来为错误推荐辩护。四种防御措施均不足以应对：怀疑提示词可能像推理一样加剧脆弱性，两种共识过滤器有压制合法产品的风险，可信度重排序虽对所有模型均有帮助，但仅能消除六分之一的虚假产品。我们在https://github.com/leoluolol/forge-benchmark发布FORGE基准和评估代码。

English

Search-augmented LLMs increasingly mediate everyday consumer recommendations by retrieving live web content. This creates a new risk: LLM recommenders may consume web content that Generative Engine Optimization (GEO) operators have polluted to mislead them. We ask: to what extent do they become unwitting promoters of fake products? We introduce FORGE (Fake Online Recommendations in Generative Environments), which locally rewrites real products in a frozen set of retrieved web pages into fake ones and measures how often the LLM recommends the fake product, across 225 real products in 15 categories and 5 consumer scenarios. Across 12 commercial and open-weights LLMs, all models are vulnerable: a single polluted page yields fooled rates of up to 27%, while the full top-3 replacement raises this to 73.8%. Vulnerability varies across categories, increasing when models lack stable prior knowledge of the products. Reasoning does not mitigate this vulnerability; instead, it often generates spurious social proof to justify false recommendations. None of the four defenses is adequate: the skepticism prompt can exacerbate vulnerability much like reasoning, the two consensus filters risk suppressing legitimate products, and credibility re-ranking helps every model but removes only a sixth of the fakes. We release the FORGE benchmark and the evaluation code at https://github.com/leoluolol/forge-benchmark.