MUSE：面向大型语言模型多模态统一安全性的运行中心化评估平台

MUSE: A Run-Centric Platform for Multimodal Unified Safety Evaluation of Large Language Models

March 3, 2026

作者: Zhongxi Wang, Yueqian Lin, Jingyang Zhang, Hai Helen Li, Yiran Chen

cs.AI

摘要

当前大语言模型的安全评估与红队测试仍以文本为中心，现有框架缺乏系统化检验对齐能力能否泛化至音频、图像及视频输入的基础设施。我们提出MUSE（多模态统一安全评估平台），这一开源、以运行为核心的平台将自动跨模态载荷生成、三种多轮攻击算法（Crescendo、PAIR、Violent Durian）、供应商无关的模型路由，以及采用五级安全分类法的LLM评判器整合至基于浏览器的统一系统中。双指标框架区分了硬性攻击成功率（仅含完全服从）与软性ASR（包含部分服从），可捕捉二元指标所遗漏的部分信息泄露。为探究对齐能力是否跨越模态边界泛化，我们引入轮次间模态切换技术，通过每轮次模态轮换增强多轮攻击效果。在四个供应商的六款多模态大语言模型上的实验表明：针对单轮拒绝率接近完美的模型，多轮攻击策略可实现90-100%的ASR；ITMS虽未在已饱和的基线上统一提升最终ASR，但通过瓦解早期轮次防御加速了攻击收敛；消融实验揭示模态影响方向具有模型家族特异性而非普适性，这凸显了需开展供应商感知的跨模态安全测试。

English

Safety evaluation and red-teaming of large language models remain predominantly text-centric, and existing frameworks lack the infrastructure to systematically test whether alignment generalizes to audio, image, and video inputs. We present MUSE (Multimodal Unified Safety Evaluation), an open-source, run-centric platform that integrates automatic cross-modal payload generation, three multi-turn attack algorithms (Crescendo, PAIR, Violent Durian), provider-agnostic model routing, and an LLM judge with a five-level safety taxonomy into a single browser-based system. A dual-metric framework distinguishes hard Attack Success Rate (Compliance only) from soft ASR (including Partial Compliance), capturing partial information leakage that binary metrics miss. To probe whether alignment generalizes across modality boundaries, we introduce Inter-Turn Modality Switching (ITMS), which augments multi-turn attacks with per-turn modality rotation. Experiments across six multimodal LLMs from four providers show that multi-turn strategies can achieve up to 90-100% ASR against models with near-perfect single-turn refusal. ITMS does not uniformly raise final ASR on already-saturated baselines, but accelerates convergence by destabilizing early-turn defenses, and ablation reveals that the direction of modality effects is model-family-specific rather than universal, underscoring the need for provider-aware cross-modal safety testing.