TextGenSHAP：在生成长文档的文本生成中实现可扩展的事后解释

TextGenSHAP: Scalable Post-hoc Explanations in Text Generation with Long Documents

December 3, 2023

作者: James Enouen, Hootan Nakhost, Sayna Ebrahimi, Sercan O Arik, Yan Liu, Tomas Pfister

cs.AI

摘要

大型语言模型（LLMs）由于其日益准确的响应和连贯的推理能力，在实际应用中引起了极大的兴趣。由于它们作为黑匣子，使用复杂的推理过程处理其输入，对于为LLMs生成的内容提供可扩展和忠实解释的需求将不可避免地增长。过去十年里，神经网络模型的可解释性已经取得了重大进展。其中，事后解释方法，特别是Shapley值，已被证明对解释深度学习模型非常有效。然而，在为LLMs扩展Shapley值时存在重大挑战，特别是在处理包含数千个标记和自回归生成的输出序列的长输入上。此外，如何有效利用生成的解释来提高LLMs性能通常是不明确的。在本文中，我们介绍了TextGenSHAP，这是一种高效的事后解释方法，结合了LM特定的技术。我们证明，与传统的Shapley值计算相比，这将大大提高速度，将处理时间从几小时缩短到几分钟，用于标记级别的解释，甚至仅需几秒用于文档级别的解释。此外，我们展示了如何在两个重要场景中利用实时Shapley值，通过定位重要单词和句子来更好地理解长文档问答；并通过增强所选段落的准确性，从而提高现有文档检索系统的性能，最终改善最终响应。

English

Large language models (LLMs) have attracted huge interest in practical applications given their increasingly accurate responses and coherent reasoning abilities. Given their nature as black-boxes using complex reasoning processes on their inputs, it is inevitable that the demand for scalable and faithful explanations for LLMs' generated content will continue to grow. There have been major developments in the explainability of neural network models over the past decade. Among them, post-hoc explainability methods, especially Shapley values, have proven effective for interpreting deep learning models. However, there are major challenges in scaling up Shapley values for LLMs, particularly when dealing with long input contexts containing thousands of tokens and autoregressively generated output sequences. Furthermore, it is often unclear how to effectively utilize generated explanations to improve the performance of LLMs. In this paper, we introduce TextGenSHAP, an efficient post-hoc explanation method incorporating LM-specific techniques. We demonstrate that this leads to significant increases in speed compared to conventional Shapley value computations, reducing processing times from hours to minutes for token-level explanations, and to just seconds for document-level explanations. In addition, we demonstrate how real-time Shapley values can be utilized in two important scenarios, providing better understanding of long-document question answering by localizing important words and sentences; and improving existing document retrieval systems through enhancing the accuracy of selected passages and ultimately the final responses.