从洞察到行动：可解释性和分析对自然语言处理的影响研究

From Insights to Actions: The Impact of Interpretability and Analysis Research on NLP

June 18, 2024

作者: Marius Mosbach, Vagrant Gautam, Tomás Vergara-Browne, Dietrich Klakow, Mor Geva

cs.AI

摘要

可解释性和分析（IA）研究是自然语言处理（NLP）领域内一个日益发展的子领域，旨在深入了解NLP系统和方法的行为或内部运作。尽管该子领域引起了越来越多的关注，但普遍的批评是缺乏可操作的见解，因此对NLP的影响有限。在本文中，我们旨在量化IA研究对更广泛的NLP领域的影响。我们采用混合方法分析：（1）从2018年至2023年在ACL和EMNLP会议上发表的所有论文构建的包含185K+篇论文的引文图，以及（2）对NLP社区的138名成员进行的调查。我们的定量结果显示，IA工作在IA之外得到了很好的引用，并且在NLP引文图中处于核心位置。通过对调查回复和手动标注的556篇论文的定性分析，我们发现NLP研究人员借鉴了IA工作的发现，并认为这对NLP的进展、多个子领域至关重要，并依赖其发现和术语来进行自己的工作。许多新颖的方法是基于IA发现提出的，并受其影响深远，但具有高影响力的非IA工作引用了IA的发现，但并非受其驱动。最后，我们总结了当今IA工作中的缺失，并提出了行动号召，为IA研究的更有影响力的未来铺平道路。

English

Interpretability and analysis (IA) research is a growing subfield within NLP with the goal of developing a deeper understanding of the behavior or inner workings of NLP systems and methods. Despite growing interest in the subfield, a commonly voiced criticism is that it lacks actionable insights and therefore has little impact on NLP. In this paper, we seek to quantify the impact of IA research on the broader field of NLP. We approach this with a mixed-methods analysis of: (1) a citation graph of 185K+ papers built from all papers published at ACL and EMNLP conferences from 2018 to 2023, and (2) a survey of 138 members of the NLP community. Our quantitative results show that IA work is well-cited outside of IA, and central in the NLP citation graph. Through qualitative analysis of survey responses and manual annotation of 556 papers, we find that NLP researchers build on findings from IA work and perceive it is important for progress in NLP, multiple subfields, and rely on its findings and terminology for their own work. Many novel methods are proposed based on IA findings and highly influenced by them, but highly influential non-IA work cites IA findings without being driven by them. We end by summarizing what is missing in IA work today and provide a call to action, to pave the way for a more impactful future of IA research.