多模態情境安全

Multimodal Situational Safety

October 8, 2024

作者: Kaiwen Zhou, Chengzhi Liu, Xuandong Zhao, Anderson Compalas, Dawn Song, Xin Eric Wang

cs.AI

摘要

多模式大型語言模型（MLLMs）正在快速演進，展示出作為多模式助手與人類及其環境互動的令人印象深刻的能力。然而，這種增加的複雜性帶來了重大的安全問題。在本文中，我們提出了一個名為多模式情境安全（Multimodal Situational Safety）的新型安全挑戰的首次評估和分析，該挑戰探討了基於使用者或代理人所參與的具體情況而變化的安全考量。我們認為，為了安全地回應，無論是通過語言還是行動，MLLMs通常需要評估語言查詢在其相應的視覺上下文中的安全影響。為了評估這種能力，我們開發了多模式情境安全基準（MSSBench），以評估當前MLLMs的情境安全表現。該數據集包括1,820個語言查詢-圖像對，其中一半的圖像上下文是安全的，另一半是不安全的。我們還開發了一個評估框架，分析關鍵的安全方面，包括明確的安全推理、視覺理解，以及至關重要的情境安全推理。我們的研究結果顯示，當前的MLLMs在指示遵循情境中遇到這種微妙的安全問題時遇到困難，並且難以一次性應對這些情境安全挑戰，突顯了未來研究的一個關鍵領域。此外，我們開發了多代理管道來協調解決安全挑戰，這在安全性方面顯示出與原始MLLM回應相比的持續改進。代碼和數據：mssbench.github.io。

English

Multimodal Large Language Models (MLLMs) are rapidly evolving, demonstrating impressive capabilities as multimodal assistants that interact with both humans and their environments. However, this increased sophistication introduces significant safety concerns. In this paper, we present the first evaluation and analysis of a novel safety challenge termed Multimodal Situational Safety, which explores how safety considerations vary based on the specific situation in which the user or agent is engaged. We argue that for an MLLM to respond safely, whether through language or action, it often needs to assess the safety implications of a language query within its corresponding visual context. To evaluate this capability, we develop the Multimodal Situational Safety benchmark (MSSBench) to assess the situational safety performance of current MLLMs. The dataset comprises 1,820 language query-image pairs, half of which the image context is safe, and the other half is unsafe. We also develop an evaluation framework that analyzes key safety aspects, including explicit safety reasoning, visual understanding, and, crucially, situational safety reasoning. Our findings reveal that current MLLMs struggle with this nuanced safety problem in the instruction-following setting and struggle to tackle these situational safety challenges all at once, highlighting a key area for future research. Furthermore, we develop multi-agent pipelines to coordinately solve safety challenges, which shows consistent improvement in safety over the original MLLM response. Code and data: mssbench.github.io.