透過表徵錨定與語言-動作對齊實現可泛化的VLA微調

摘要

將預訓練的視覺語言模型（VLM）透過行為克隆（BC）在機器人示範資料上進行微調，已成為視覺語言動作（VLA）策略的標準做法。然而，BC微調會逐步覆蓋支撐視覺與語義泛化能力的預訓練表徵。常見的補救措施——在網路圖文資料上進行協同訓練——並無法避免此問題；它將語言與動作損失應用於不同的觀測資料，導致VLA存在語言與動作不對齊，而標準操控基準測試並未揭露此缺陷。我們提出Anchor-Align方法，在BC中增加兩個目標函數：視覺語言錨定（Vision-Language Anchoring）從凍結的VLM副本中蒸餾逐層表徵以預防此偏移；語言動作對齊（Language-Action Alignment）則將每個動作目標轉換為離散的運動方向標籤，並在同一機器人觀測資料上同時訓練語言與動作預測。在實體xArm7機器人上，針對兩種廣泛使用的VLA架構，Anchor-Align分別將真實機器人成功率從28%提升至54%、從37%提升至60%。在模擬環境大規模實驗中，我們分別在LIBERO-PRO、LIBERO-Plus與CALVIN上觀察到其對分佈外擾動（OOD）、感知穩健性及長程控制均有持續改善，顯示保留預訓練表徵與有效學習動作之間並非根本衝突。專案頁面：anchoralignvla.github.io

English

Finetuning a pretrained vision-language model (VLM) on robot demonstrations via behavior cloning (BC) has become the standard recipe for vision-language-action (VLA) policies. However, BC finetuning progressively overwrites the pretrained representations that support visual and semantic generalization. Co-training on web image-text data, a common remedy, does not prevent this; it applies language and action losses to separate observations, leaving VLAs with language-action misalignment that standard manipulation benchmarks do not expose. We propose Anchor-Align, which augments BC with two objectives: Vision-Language Anchoring distills layer-wise representations from a frozen VLM copy to prevent this drift, while Language-Action Alignment converts each action target into a discrete motion-direction label and jointly trains language and action prediction on the same robot observation. On a physical xArm7 robot, across two widely used VLA architectures, Anchor-Align improves real-robot success on both (28% to 54% and 37% to 60%). At scale in simulation, we demonstrate consistent improvements on OOD perturbations, perceptual robustness, and long-horizon control across LIBERO-PRO, LIBERO-Plus, and CALVIN, respectively, suggesting that preserving pretrained representations and effective action learning are not fundamentally at odds. Project page: anchoralignvla.github.io