Foresight：基于动作条件的世界模型潜变量的长时域机器人操作失败检测

摘要

长时域任务在现实机器人部署中很常见，但其失败检测问题仍鲜有研究。检测长时域机器人任务的失败尤为困难，因为失败起点往往模糊不清，且通常缺乏密集的时间标注。我们提出Foresight，一个利用动作条件世界模型的潜在表征来监控操作轨迹的失败检测框架。Foresight仅通过最终任务级成功或失败标签进行训练。通过利用预测性的世界模型嵌入，我们的方法为不同策略提供了统一的失败检测框架。我们进一步使用函数共形预测（FCP）自适应地校准检测阈值。我们基于LIBERO-Long、ManiSkill-Long和BEHAVIOR-1K仿真环境中的最先进视觉-语言-动作策略评估了Foresight，并将其与现有最优失败检测方法进行比较，同时在ReactorX-200机械臂上执行三个长时域任务、在Franka机械臂上执行一个任务进行实物验证。结果表明，动作条件世界模型嵌入为长时域操作中的可靠失败监控提供了可扩展的表征。

English

Long-horizon tasks are common in real-world robotic deployments, yet failure detection for such tasks remains underexplored. Detecting failures in long-horizon robotic tasks is particularly challenging because failure onset is often ambiguous and dense temporal annotations are typically unavailable. We present Foresight, a failure detection framework that monitors manipulation trajectories using latent representations from an action-conditioned world model. Foresight is trained using only final task-level success or failure labels. By leveraging predictive world-model embeddings, our method provides a unified framework for failure detection across different policies. We further use functional conformal prediction (FCP) to calibrate detection thresholds adaptively. We evaluate Foresight with state-of-the-art vision-language-action policies in simulation on LIBERO-Long, ManiSkill-Long, and BEHAVIOR-1K, compare it against state-of-the-artfailure detection methods, and validate it on real robots with three long-horizon tasks on a ReactorX-200 arm and one task on a Franka arm. Our results suggest that action-conditioned world-model embeddings provide a scalable representation for reliable failure monitoring in long-horizon manipulation.