機器人需要的遠不止VLA與世界模型

摘要

通用機器人智慧通常被視為一種策略規模化的問題：收集更多機器人示範數據、訓練更大規模的視覺-語言-行動（VLA）模型，並期望獲得更廣泛的泛化能力。在這篇立場論文中，我們認為此框架並不完整。核心瓶頸不僅在於策略學習，更在於缺乏將世界豐富的非結構化行為數據轉化為具體機器人監督訊號的機制。人類動作、網路影片、模擬推演與互動示範中蘊含了大量關於任務、目標、接觸、失敗及物理限制的資訊，然而這些資訊大多無法直接被機器人策略使用，因為它們缺少具身特定的動作標籤、任務語義及獎勵結構。我們指出下一代機器人系統所需的核心缺失組件：用於自動標註非結構化行為的數據介面、將人類動作重新對應至機器人行動的具身介面、基於物理模型的三維推理世界模型介面，以及從影片與語言推斷任務進展與成功的獎勵介面。我們回顧了機器人基礎模型、跨具身數據集、從影片中學習、世界模型及獎勵建模等領域的最新進展，並提出一套研究議程，旨在建構不僅能從機器人示範中學習，更能從更廣泛的物理世界中學習的機器人系統。

English

Generalist robot intelligence is often framed as a policy-scaling problem: collect more robot demonstrations, train larger Vision-Language-Action (VLA) models, and expect broader generalisation. In this position paper, we argue that this framing is incomplete. The central bottleneck is not only policy learning, but the absence of mechanisms that convert the world's abundant unstructured behavioural data into grounded robot supervision. Human motion, internet video, simulation rollouts, and interactive demonstrations contain rich information about tasks, goals, contacts, failures, and physical constraints, yet most of this information is not directly usable by robot policies because it lacks embodiment-specific action labels, task semantics, and reward structure. We identify four missing components for the next generation of robotics: data interfaces for autolabelling unstructured behaviour, embodiment interfaces for retargeting human motion to robot actions, world-model interfaces for physics-grounded 3D reasoning, and reward interfaces for inferring task progress and success from video and language. We survey recent progress in robot foundation models, cross-embodiment datasets, learning from video, world models, and reward modelling, and propose a research agenda for building robotics systems that can learn not only from robot demonstrations, but from the broader physical world.