HarmProfile：描繪前沿大型語言模型中的有害分布

摘要

前沿大型語言模型（LLM）的安全評估，長期以來大多將有害生成視為攻擊結果，而非分析對象。因此，我們對模型在行為異常時所產生的有害輸出知之甚少，部分原因在於難以取得大規模且高品質的前沿 LLM 異常行為資料。為填補此一缺口，我們提出 HarmProfile，這是一個以內容為中心的基準資料集，收集了跨越多種危害類別與模型家族之模型異常行為，並將由此產生的有害輸出分布定義為模型層級的風險輪廓。其前提在於：正如語言行為可從語料庫中的話語加以刻畫，模型風險也可由其安全失敗的內容、嚴重程度與變異來描述。HarmProfile 收錄了來自 23 個前沿 LLM、涵蓋 13 個模型家族、超過 80,000 個經驗證的產出，並組織為 15 個危害類別與 57 個子類別。利用此語料庫，我們發現前沿 LLM 確實會大規模產生有害內容，但同時展現出各自的風險輪廓；危害性與多樣性皆隨模型能力增強而增加，這顯示前沿 LLM 或許看似安全，但在其對齊表層之下，卻潛藏著日益危險的知識。我們的原始碼可在 https://github.com/fresh-ma/HarmProfile 取得。

English

Frontier large language models (LLMs) safety evaluation has largely treated harmful generation as an attack outcome rather than as an object of analysis. Consequently, little is known about the harmful outputs produced during model misbehavior, partly because large-scale, high-quality collections of frontier-LLM misbehavior are difficult to obtain. To address this gap, we introduce HarmProfile, a content-centric benchmark dataset that collects model misbehavior across diverse harm categories and model families, and defines the resulting harmful-output distribution as a model-level risk profile. The premise is that, just as linguistic behavior can be characterized from an utterance corpus, model risk can be characterized from the content, severity, and variation of its safety failures. HarmProfile contains over 80,000 validated artifacts from 23 frontier LLMs across 13 model families, organized into 15 harm categories and 57 subcategories. Using this corpus, we find that frontier LLMs reliably produce harmful content at scale, yet exhibit distinct risk profiles; both harmfulness and diversity grow with model capability, suggesting that frontier LLMs may appear safe yet harbor increasingly dangerous knowledge beneath the alignment surface. Our source code is available at https://github.com/fresh-ma/HarmProfile .