InSight：透過可操控的VLA進行自我引導的技能獲取

摘要

視覺-語言-動作（VLA）模型能從示範中學習操作技能，但其能力受限於訓練資料中的技能範圍。我們提出 InSight 框架，透過使 VLA 在原始動作層級（例如「將夾爪移至碗邊」、「向上提起」、「傾倒瓶子」）具備可操控性，從而實現自主技能獲取。InSight 包含兩個主要階段：（1）自動化分割流程，透過 VLM 計畫分解與末端執行器位姿，將示範資料分割為帶標籤的原始動作，使 VLA 能操控這些原始動作；（2）VLM 引導的資料飛輪，用於辨識完成新任務所需的缺失原始動作，並以 VLM 提出的低階控制自主嘗試這些缺失原始動作的示範，再將成功的示範自動標記、儲存並整合至 VLA 訓練集。我們在模擬環境與真實世界操作任務（包括翻轉方塊、關閉抽屜、掃地、扭轉及傾倒）中評估 InSight，且這些目標技能皆未使用任何人類示範。一旦習得，這些原始動作可組合執行新穎的長時域任務，無需額外的人類示範。我們的研究結果顯示，原始動作的可操控性為 VLA 策略的持續技能獲取提供了實用基礎。專案網站：https://insight-vla.github.io。

English

Vision-language-action (VLA) models can learn manipulation skills from demonstrations, but their capabilities are bounded by the skills in the training data. We present InSight, a framework that unlocks autonomous skill acquisition by rendering VLAs steerable at the primitive-action level (e.g., "move gripper to the bowl", "lift upward", "pour the bottle"). InSight consists of two primary stages: (1) an automated segmentation pipeline that partitions demonstrations into labeled primitives via VLM plan decomposition and end-effector poses to enable VLA primitive steerability, and (2) a VLM-guided data flywheel that identifies missing primitives required to accomplish a novel task, autonomously attempts demonstrations of the missing primitives with VLM-proposed low-level control, and automatically labels, stores, and integrates successful demonstrations into the VLA training set. We evaluate InSight across simulation and real-world manipulation tasks, including block flipping, drawer closing, sweeping, twisting, and pouring, without any human demonstrations of these target skills. Once learned, these primitives can be composed to execute novel, long-horizon tasks without additional human demonstrations. Our findings demonstrate that primitive steerability provides a practical foundation for continual skill acquisition in VLA policies. Project website: https://insight-vla.github.io.