将CVE映射到MITRE ATT&CK技术：一个精心策划的金标准分类器及LLM辅助标签扩展的局限性

摘要

我们提出了一种可复现的流水线，用于从自由文本的漏洞描述中，将通用漏洞与暴露（CVE）映射到MITRE ATT&CK企业技术。我们不再依赖CWE->CAPEC->ATT&CK的推导链（我们量化了该链中表格扩展的虚假现象），而是在由MITRE威胁知情防御中心的专家映射构建的1207个CVE策划黄金数据集上，训练了一个多标签分类器。与零样本嵌入相似性基线相比，该模型在recall@5上大约翻倍，并改进了所有排名指标。随后，我们研究了通过大语言模型辅助标注能否扩展黄金数据集。初步实验得出了矛盾的结论：单次运行表明性能下降，而对五个随机种子取平均值则显示轻微提升。然而，独立复现和扩展规模研究（增加100至984个CVE）表明，这种表面上的改进是评估假象。大语言模型生成的标签与专家标注的一致性约为0.39，在任何扩展规模下均未提供可靠改进，并在添加约1000个CVE时降低了稀有技术覆盖率（macro-F1下降0.04）。根本原因在于评估噪声。在较小的测试集上选择检查点实际上是在多个噪声评估上最大化，导致相同运行之间recall@5差异可达0.05。采用基于验证集划分检查点选择的修正协议后，仅使用黄金数据的模型达到recall@5为（0.673 ± 0.019），且重复关键实验确认了大语言模型扩展的零结果。最终规模研究表明，额外的专家策划数据能持续提升性能，而大语言模型标注的数据则不能，这表明分类器的瓶颈在于标签质量而非数据集规模。所有数据集、模型、代码和训练日志均已公开发布。

English

We present a reproducible pipeline for mapping Common Vulnerabilities and Exposures (CVEs) to MITRE ATT&CK Enterprise techniques from free-text vulnerability descriptions. Rather than relying on the CWE->CAPEC->ATT&CK derivation chain, whose table-expansion artifacts we quantify, we train a multi-label classifier on a curated gold dataset of 1,207 CVEs from expert MITRE Center for Threat-Informed Defense mappings. The resulting model approximately doubles recall@5 compared with a zero-shot embedding-similarity baseline and improves every ranking metric. We then investigate whether LLM-assisted labeling can extend the gold dataset. Initial experiments suggest contradictory conclusions: a single run indicates degraded performance, while averaging over five random seeds suggests a small gain. However, an independent replication and an expansion-size study (100--984 additional CVEs) show that the apparent improvement is an evaluation artifact. LLM-generated labels, with approximately 0.39 agreement with expert annotations, provide no reliable improvement at any expansion size and reduce rare-technique coverage at around 1,000 added CVEs (macro-F1 decreases by 0.04). The root cause is evaluation noise. Selecting checkpoints on a small test split effectively maximizes over many noisy evaluations, producing recall@5 differences of up to 0.05 between otherwise identical runs. Using a corrected protocol based on validation-split checkpoint selection, the gold-only model achieves recall@5 of (0.673 \pm 0.019), and repeating the decisive experiment confirms the null result for LLM expansion. A final scaling study shows that additional expert-curated data consistently improves performance, whereas LLM-labeled data does not, indicating that the classifier is limited by label quality rather than dataset size. All datasets, models, code, and training logs are publicly released.