前瞻：使用动作条件世界模型潜变量的长程机器人操作故障检测

摘要

長期時域任務在真實世界的機器人部署中相當常見，然而針對此類任務的故障檢測仍鮮有探討。檢測長期機器人任務中的故障尤其具有挑戰性，因為故障的發生往往難以明確界定，且密集的時間標註通常難以取得。我們提出Foresight，這是一個故障檢測框架，透過動作條件化世界模型的潛在表徵來監控操作軌跡。Foresight僅使用最終任務層級的成功或失敗標籤進行訓練。藉由利用預測性世界模型嵌入，我們的方法為不同策略下的故障檢測提供了統一的框架。我們進一步採用函數共形預測來自適應地校準檢測閾值。我們在模擬環境中，針對LIBERO-Long、ManiSkill-Long和BEHAVIOR-1K，使用最先進的視覺-語言-動作策略來評估Foresight，並與現有的頂尖故障檢測方法進行比較，同時在真實機器人上，於ReactorX-200機械臂執行三項長期任務及Franka機械臂執行一項任務進行驗證。結果顯示，動作條件化世界模型嵌入能為長期操作中的可靠故障監控提供可擴展的表徵。

English

Long-horizon tasks are common in real-world robotic deployments, yet failure detection for such tasks remains underexplored. Detecting failures in long-horizon robotic tasks is particularly challenging because failure onset is often ambiguous and dense temporal annotations are typically unavailable. We present Foresight, a failure detection framework that monitors manipulation trajectories using latent representations from an action-conditioned world model. Foresight is trained using only final task-level success or failure labels. By leveraging predictive world-model embeddings, our method provides a unified framework for failure detection across different policies. We further use functional conformal prediction (FCP) to calibrate detection thresholds adaptively. We evaluate Foresight with state-of-the-art vision-language-action policies in simulation on LIBERO-Long, ManiSkill-Long, and BEHAVIOR-1K, compare it against state-of-the-artfailure detection methods, and validate it on real robots with three long-horizon tasks on a ReactorX-200 arm and one task on a Franka arm. Our results suggest that action-conditioned world-model embeddings provide a scalable representation for reliable failure monitoring in long-horizon manipulation.