HarmProfile：刻画前沿大语言模型中的有害分布

摘要

前沿大语言模型（LLM）的安全评估在很大程度上将有害生成视为一种攻击结果，而非分析对象。因此，人们对模型在不当行为期间产生的有害输出知之甚少，部分原因在于大规模、高质量的前沿LLM不当行为数据集难以获取。为弥补这一空白，我们引入了HarmProfile——一个以内容为中心的基准数据集，该数据集收集了跨多样危害类别和模型家族的模型不当行为，并将由此产生的有害输出分布定义为模型级风险画像。其前提是，正如语言行为可以通过话语语料库来刻画，模型风险也可以通过其安全失败的内容、严重程度和变化性来刻画。HarmProfile包含来自13个模型家族的23个前沿LLM的超过80,000个经验证的产物，组织为15个危害类别和57个子类别。利用该语料库，我们发现前沿LLM确实能够大规模产生有害内容，但表现出各自不同的风险画像；危害性和多样性均随模型能力而增长，这表明前沿LLM可能看似安全，却在对齐的表象之下潜藏着日益危险的知识。我们的源代码可在 https://github.com/fresh-ma/HarmProfile 获取。

English

Frontier large language models (LLMs) safety evaluation has largely treated harmful generation as an attack outcome rather than as an object of analysis. Consequently, little is known about the harmful outputs produced during model misbehavior, partly because large-scale, high-quality collections of frontier-LLM misbehavior are difficult to obtain. To address this gap, we introduce HarmProfile, a content-centric benchmark dataset that collects model misbehavior across diverse harm categories and model families, and defines the resulting harmful-output distribution as a model-level risk profile. The premise is that, just as linguistic behavior can be characterized from an utterance corpus, model risk can be characterized from the content, severity, and variation of its safety failures. HarmProfile contains over 80,000 validated artifacts from 23 frontier LLMs across 13 model families, organized into 15 harm categories and 57 subcategories. Using this corpus, we find that frontier LLMs reliably produce harmful content at scale, yet exhibit distinct risk profiles; both harmfulness and diversity grow with model capability, suggesting that frontier LLMs may appear safe yet harbor increasingly dangerous knowledge beneath the alignment surface. Our source code is available at https://github.com/fresh-ma/HarmProfile .