UniversalNER：从大型语言模型中进行有针对性的精炼，用于开放式命名实体识别

UniversalNER: Targeted Distillation from Large Language Models for Open Named Entity Recognition

August 7, 2023

作者: Wenxuan Zhou, Sheng Zhang, Yu Gu, Muhao Chen, Hoifung Poon

cs.AI

摘要

大型语言模型（LLMs）展示了出色的泛化能力，例如理解任意实体和关系。指导调整已被证明对将LLMs提炼为更具成本效益的模型（如Alpaca和Vicuna）非常有效。然而，这样的学生模型在下游应用中仍然远远落后于原始LLMs。在本文中，我们探讨了针对性提炼和以任务为中心的指导调整，以训练能在广泛应用类别（如开放信息提取）中表现出色的学生模型。通过以命名实体识别（NER）为案例研究，我们展示了如何将ChatGPT提炼为更小的UniversalNER模型，用于开放NER。为了评估，我们汇编了迄今为止最大的NER基准，包括来自9个不同领域（如生物医学、编程、社交媒体、法律、金融）的43个数据集。在不使用任何直接监督的情况下，UniversalNER在成千上万种实体类型中实现了出色的NER准确性，平均超过Alpaca和Vicuna等通用指导调整模型30个绝对F1点。只使用极少量参数，UniversalNER不仅具备ChatGPT在识别任意实体类型方面的能力，而且在NER准确性方面平均超过7-9个绝对F1点。值得注意的是，UniversalNER甚至在很大程度上胜过了最先进的多任务指导调整系统（如InstructUIE），后者使用了受监督的NER示例。我们还进行了彻底的消融研究，以评估我们提炼方法中各个组成部分的影响。我们将发布提炼配方、数据和UniversalNER模型，以促进未来针对性提炼研究。

English

Large language models (LLMs) have demonstrated remarkable generalizability, such as understanding arbitrary entities and relations. Instruction tuning has proven effective for distilling LLMs into more cost-efficient models such as Alpaca and Vicuna. Yet such student models still trail the original LLMs by large margins in downstream applications. In this paper, we explore targeted distillation with mission-focused instruction tuning to train student models that can excel in a broad application class such as open information extraction. Using named entity recognition (NER) for case study, we show how ChatGPT can be distilled into much smaller UniversalNER models for open NER. For evaluation, we assemble the largest NER benchmark to date, comprising 43 datasets across 9 diverse domains such as biomedicine, programming, social media, law, finance. Without using any direct supervision, UniversalNER attains remarkable NER accuracy across tens of thousands of entity types, outperforming general instruction-tuned models such as Alpaca and Vicuna by over 30 absolute F1 points in average. With a tiny fraction of parameters, UniversalNER not only acquires ChatGPT's capability in recognizing arbitrary entity types, but also outperforms its NER accuracy by 7-9 absolute F1 points in average. Remarkably, UniversalNER even outperforms by a large margin state-of-the-art multi-task instruction-tuned systems such as InstructUIE, which uses supervised NER examples. We also conduct thorough ablation studies to assess the impact of various components in our distillation approach. We will release the distillation recipe, data, and UniversalNER models to facilitate future research on targeted distillation.