RoboTALES：透過任務對齊的模擬未來進行推理引導的機器人策略學習

摘要

預訓練的影片生成模型在視覺運動控制中作為骨幹極具潛力，但它們想像的未來情境往往偏離任務意圖，且無法可靠地以動作條件為基礎。因此，這些模型在規劃或策略提取上不易使用。為克服這些限制，我們提出RoboTALES，一個學習任務對齊模擬未來情境，並利用其訓練機器人策略的單階段框架。我們的方法包含兩項關鍵創新：（1）一個基於層級LLM的規劃器，將複雜任務分解為一系列子目標，以引導模型的想像；以及（2）一個基於VLM的評判器，評估這些「想像的」未來情境，並利用基於獎勵的反饋，使模型的內部表徵專注於目標。透過將影片生成器錨定在抽象推理上，我們能產出時間一致的情境展開與更連貫的動作。我們在RoboCasa與LIBERO10的多樣化操作任務上評估RoboTALES，結果顯示我們的方法持續優於現有方法，尤其在長程任務中表現更佳。我們的程式碼與模型已公開於https://github.com/hananshafi/RoboTALES。

English

Pretrained video generative models are promising backbones for visuomotor control, but their imagined futures often drift from task intent and are not reliably action-conditional. As a result, these models can be difficult to use for planning or policy extraction. To address these limitations, we propose RoboTALES, a single-stage framework that learns task-aligned simulated futures and uses them to train robot policies. Our approach introduces two key innovations: (1) a hierarchical LLM-based planner that breaks complex tasks into a sequence of subgoals to guide the model's imagination; and (2) a VLM-based critic that evaluates these ``imagined'' futures and uses reward-based feedback to keep the model's internal representations focused on the goal. By anchoring the video generator in abstract reasoning, we produce temporally consistent rollouts and more coherent actions. We evaluate RoboTALES on diverse manipulation tasks from RoboCasa and LIBERO10, and show that our method consistently outperforms existing methods, especially in long-horizon tasks. Our code and models are publicly available at https://github.com/hananshafi/RoboTALES.