通过对比触发学习对多模态大语言模型具身决策实施视觉后门攻击

Visual Backdoor Attacks on MLLM Embodied Decision Making via Contrastive Trigger Learning

October 31, 2025

作者: Qiusi Zhan, Hyeonjeong Ha, Rui Yang, Sirui Xu, Hanyang Chen, Liang-Yan Gui, Yu-Xiong Wang, Huan Zhang, Heng Ji, Daniel Kang

cs.AI

摘要

多模态大语言模型（MLLMs）通过实现基于视觉输入的直接感知、推理和任务导向行动规划，推动了具身智能体的发展。然而，这种视觉驱动的具身智能体也带来了新的攻击面：视觉后门攻击。此类攻击中，智能体在场景未出现视觉触发器时表现正常，一旦触发器出现便会持续执行攻击者预设的多步策略。我们提出BEAT框架，首次实现基于环境物体作为触发器的MLLM具身智能体视觉后门注入。与文本触发器不同，物体触发器会因视角和光照条件产生巨大差异，导致可靠植入困难。BEAT通过以下方式解决该挑战：（1）构建覆盖多样化场景、任务及触发器布局的训练集，使智能体充分接触触发器变异；（2）引入两阶段训练方案，先进行监督微调（SFT），再采用新颖的对比触发器学习（CTL）。CTL将触发器判别构建为含触发器与无触发器输入的偏好学习问题，通过显式锐化决策边界确保精准的后门激活。在多种具身智能体基准测试和MLLMs中，BEAT实现了高达80%的攻击成功率，同时保持优异的正常任务性能，并能可靠泛化至分布外触发器布局。值得注意的是，在有限后门数据下，CTL相较传统SFT将后门激活准确率最高提升39%。这些发现揭示了基于MLLM的具身智能体存在重大却未被探索的安全风险，凸显了实际部署前构建鲁棒防御机制的必要性。

English

Multimodal large language models (MLLMs) have advanced embodied agents by enabling direct perception, reasoning, and planning task-oriented actions from visual inputs. However, such vision driven embodied agents open a new attack surface: visual backdoor attacks, where the agent behaves normally until a visual trigger appears in the scene, then persistently executes an attacker-specified multi-step policy. We introduce BEAT, the first framework to inject such visual backdoors into MLLM-based embodied agents using objects in the environments as triggers. Unlike textual triggers, object triggers exhibit wide variation across viewpoints and lighting, making them difficult to implant reliably. BEAT addresses this challenge by (1) constructing a training set that spans diverse scenes, tasks, and trigger placements to expose agents to trigger variability, and (2) introducing a two-stage training scheme that first applies supervised fine-tuning (SFT) and then our novel Contrastive Trigger Learning (CTL). CTL formulates trigger discrimination as preference learning between trigger-present and trigger-free inputs, explicitly sharpening the decision boundaries to ensure precise backdoor activation. Across various embodied agent benchmarks and MLLMs, BEAT achieves attack success rates up to 80%, while maintaining strong benign task performance, and generalizes reliably to out-of-distribution trigger placements. Notably, compared to naive SFT, CTL boosts backdoor activation accuracy up to 39% under limited backdoor data. These findings expose a critical yet unexplored security risk in MLLM-based embodied agents, underscoring the need for robust defenses before real-world deployment.