LVLM-Intrepret：用於大型視覺語言模型的可解釋性工具

LVLM-Intrepret: An Interpretability Tool for Large Vision-Language Models

April 3, 2024

作者: Gabriela Ben Melech Stan, Raanan Yehezkel Rohekar, Yaniv Gurwicz, Matthew Lyle Olson, Anahita Bhiwandiwalla, Estelle Aflalo, Chenfei Wu, Nan Duan, Shao-Yen Tseng, Vasudev Lal

cs.AI

摘要

在人工智慧快速發展的領域中，多模式大型語言模型正成為一個重要的研究領域。這些模型結合了各種形式的數據輸入，因此變得越來越受歡迎。然而，理解它們的內部機制仍然是一個複雜的任務。在可解釋性工具和機制的領域中已經取得了許多進展，但仍有許多待探索之處。在這項工作中，我們提出了一個新穎的互動應用程序，旨在理解大型視覺語言模型的內部機制。我們設計的界面旨在提高圖像補丁的可解釋性，這對於生成答案至關重要，並評估語言模型在圖像中基於其輸出的有效性。通過我們的應用程序，用戶可以系統地研究模型，揭示系統的局限性，為提升系統能力鋪平道路。最後，我們提出了一個案例研究，展示了我們的應用程序如何幫助理解一個流行的大型多模式模型LLaVA中的失敗機制。

English

In the rapidly evolving landscape of artificial intelligence, multi-modal large language models are emerging as a significant area of interest. These models, which combine various forms of data input, are becoming increasingly popular. However, understanding their internal mechanisms remains a complex task. Numerous advancements have been made in the field of explainability tools and mechanisms, yet there is still much to explore. In this work, we present a novel interactive application aimed towards understanding the internal mechanisms of large vision-language models. Our interface is designed to enhance the interpretability of the image patches, which are instrumental in generating an answer, and assess the efficacy of the language model in grounding its output in the image. With our application, a user can systematically investigate the model and uncover system limitations, paving the way for enhancements in system capabilities. Finally, we present a case study of how our application can aid in understanding failure mechanisms in a popular large multi-modal model: LLaVA.