SAFE：視覺-語言-動作模型的多任務故障檢測

SAFE: Multitask Failure Detection for Vision-Language-Action Models

June 11, 2025

作者: Qiao Gu, Yuanliang Ju, Shengxiang Sun, Igor Gilitschenski, Haruki Nishimura, Masha Itkina, Florian Shkurti

cs.AI

摘要

儘管視覺-語言-動作模型（VLAs）在多樣化的操作任務中展現出令人期待的機器人行為，但在面對全新任務時，其開箱即用的成功率仍有限。為了讓這些策略能夠安全地與環境互動，我們需要一個故障檢測器，它能及時發出警報，使機器人能夠停止、回退或尋求幫助。然而，現有的故障檢測器僅在一個或少數特定任務上進行訓練和測試，而VLAs則要求檢測器能夠泛化，並在未見任務和新環境中也能檢測到故障。本文中，我們提出了多任務故障檢測問題，並針對如VLAs這樣的通用機器人策略，提出了SAFE故障檢測器。我們分析了VLA的特徵空間，發現VLAs具備足夠的高層次知識來判斷任務的成功與失敗，這些知識在不同任務間具有通用性。基於這一洞察，我們設計了SAFE，讓其從VLA的內部特徵中學習，並預測一個指示任務失敗可能性的單一標量。SAFE在成功和失敗的執行軌跡上進行訓練，並在未見任務上進行評估。SAFE兼容多種策略架構，我們在OpenVLA、pi_0及pi_0-FAST上，於模擬和真實環境中進行了廣泛測試。通過與多種基線方法的比較，我們展示了SAFE在故障檢測性能上達到了業界領先水平，並利用合規預測實現了準確性與檢測時間之間的最佳平衡。更多定性結果可訪問https://vla-safe.github.io/。

English

While vision-language-action models (VLAs) have shown promising robotic behaviors across a diverse set of manipulation tasks, they achieve limited success rates when deployed on novel tasks out-of-the-box. To allow these policies to safely interact with their environments, we need a failure detector that gives a timely alert such that the robot can stop, backtrack, or ask for help. However, existing failure detectors are trained and tested only on one or a few specific tasks, while VLAs require the detector to generalize and detect failures also in unseen tasks and novel environments. In this paper, we introduce the multitask failure detection problem and propose SAFE, a failure detector for generalist robot policies such as VLAs. We analyze the VLA feature space and find that VLAs have sufficient high-level knowledge about task success and failure, which is generic across different tasks. Based on this insight, we design SAFE to learn from VLA internal features and predict a single scalar indicating the likelihood of task failure. SAFE is trained on both successful and failed rollouts, and is evaluated on unseen tasks. SAFE is compatible with different policy architectures. We test it on OpenVLA, pi_0, and pi_0-FAST in both simulated and real-world environments extensively. We compare SAFE with diverse baselines and show that SAFE achieves state-of-the-art failure detection performance and the best trade-off between accuracy and detection time using conformal prediction. More qualitative results can be found at https://vla-safe.github.io/.