经认证的最坏情况下LLM版权侵权缓解方案

Certified Mitigation of Worst-Case LLM Copyright Infringement

April 22, 2025

作者: Jingyu Zhang, Jiacan Yu, Marc Marone, Benjamin Van Durme, Daniel Khashabi

cs.AI

摘要

大型语言模型（LLMs）在预训练阶段接触受版权保护材料，引发了部署后可能无意间侵犯版权的担忧。这推动了“版权移除”方法的发展，即旨在防止模型生成与受版权保护内容高度相似的后训练策略。尽管现有的缓解措施对平均风险有一定效果，但我们发现它们忽视了最坏情况下的版权风险，这些风险体现在模型可能生成来自受版权来源的长篇逐字引用。为此，我们提出了BloomScrub，一种极其简单却极为有效的推理时方法，它提供了认证的版权移除功能。该方法通过反复交织引用检测与重写技术，来转换潜在的侵权片段。借助高效的数据草图（布隆过滤器），我们的方法能够对大规模现实世界语料库进行可扩展的版权筛查。当无法移除超过长度阈值的引用时，系统可选择不回应，从而确保风险降低。实验结果表明，BloomScrub有效降低了侵权风险，保持了实用性，并通过自适应弃权机制适应了不同严格程度的执行要求。我们的研究结果表明，轻量级的推理时方法在版权预防方面具有出人意料的效力。

English

The exposure of large language models (LLMs) to copyrighted material during pre-training raises concerns about unintentional copyright infringement post deployment. This has driven the development of "copyright takedown" methods, post-training approaches aimed at preventing models from generating content substantially similar to copyrighted ones. While current mitigation approaches are somewhat effective for average-case risks, we demonstrate that they overlook worst-case copyright risks exhibits by the existence of long, verbatim quotes from copyrighted sources. We propose BloomScrub, a remarkably simple yet highly effective inference-time approach that provides certified copyright takedown. Our method repeatedly interleaves quote detection with rewriting techniques to transform potentially infringing segments. By leveraging efficient data sketches (Bloom filters), our approach enables scalable copyright screening even for large-scale real-world corpora. When quotes beyond a length threshold cannot be removed, the system can abstain from responding, offering certified risk reduction. Experimental results show that BloomScrub reduces infringement risk, preserves utility, and accommodates different levels of enforcement stringency with adaptive abstention. Our results suggest that lightweight, inference-time methods can be surprisingly effective for copyright prevention.