深度无知：通过预训练数据筛选为开放权重大语言模型构建防篡改安全机制

Deep Ignorance: Filtering Pretraining Data Builds Tamper-Resistant Safeguards into Open-Weight LLMs

August 8, 2025

作者: Kyle O'Brien, Stephen Casper, Quentin Anthony, Tomek Korbak, Robert Kirk, Xander Davies, Ishan Mishra, Geoffrey Irving, Yarin Gal, Stella Biderman

cs.AI

摘要

开放权重AI系统提供了独特的优势，包括增强的透明度、开放的研究环境以及去中心化的访问。然而，它们容易受到篡改攻击，通过修改权重或激活值，这些攻击能高效地诱导出有害行为。目前，尚未形成一套完善的开放权重模型风险管理科学。现有的安全微调方法及其他训练后技术难以使大语言模型（LLMs）抵御超过几十步的对抗性微调。本文探讨了从训练数据中过滤涉及双重用途主题的文本，是否能防止不期望的能力出现，并作为一种更抗篡改的防护措施。我们引入了一个多阶段的可扩展数据过滤流程，并展示了其作为一种可行且有效的方法，能够最小化LLMs中生物威胁代理知识的存在。我们从零开始预训练了多个6.9B参数的模型，发现它们对多达10,000步和300M个生物威胁相关文本的对抗性微调攻击表现出显著的抵抗力——超越现有训练后基线方法一个数量级以上——且未观察到对无关能力的退化。然而，尽管过滤后的模型内部缺乏危险知识，我们发现当这些信息在上下文中提供时（例如，通过搜索工具增强），模型仍能利用此类信息，这表明需要一种深度防御策略。总体而言，这些发现有助于确立预训练数据筛选作为开放权重AI系统防御体系中的一个有前景的层次。

English

Open-weight AI systems offer unique benefits, including enhanced transparency, open research, and decentralized access. However, they are vulnerable to tampering attacks which can efficiently elicit harmful behaviors by modifying weights or activations. Currently, there is not yet a robust science of open-weight model risk management. Existing safety fine-tuning methods and other post-training techniques have struggled to make LLMs resistant to more than a few dozen steps of adversarial fine-tuning. In this paper, we investigate whether filtering text about dual-use topics from training data can prevent unwanted capabilities and serve as a more tamper-resistant safeguard. We introduce a multi-stage pipeline for scalable data filtering and show that it offers a tractable and effective method for minimizing biothreat proxy knowledge in LLMs. We pretrain multiple 6.9B-parameter models from scratch and find that they exhibit substantial resistance to adversarial fine-tuning attacks on up to 10,000 steps and 300M tokens of biothreat-related text -- outperforming existing post-training baselines by over an order of magnitude -- with no observed degradation to unrelated capabilities. However, while filtered models lack internalized dangerous knowledge, we find that they can still leverage such information when it is provided in context (e.g., via search tool augmentation), demonstrating a need for a defense-in-depth approach. Overall, these findings help to establish pretraining data curation as a promising layer of defense for open-weight AI systems.