重新思考面向VLA初始化的VLM表示

摘要

视觉-语言-动作（VLA）模型广泛采用预训练的视觉-语言模型（VLM）作为策略主干，但尚不清楚何种预训练VLM表示对VLA初始化有效。本文从三个维度将VLA初始化作为受控表示设计问题进行研究：能力级别的具身化VQA监督、参数更新策略以及机器人数据预训练。实验表明，原始预训练VLM表示是动作性能的关键来源。然而，具身化VQA适配并未带来一致性的性能提升：其收益取决于下游瓶颈，且不同能力领域的增益并非简单叠加。在更新策略方面，LoRA相比全量微调能提供更可靠的初始化效果，这表明过度重塑预训练表示会削弱VLA初始化能力。机器人数据预训练可进一步改善VLA初始化，其中基于LoRA的分阶段训练策略产生的变体效果最佳。综合这些发现可知，有效的VLM-to-VLA适配应在注入与动作相关的具身和机器人轨迹信号的同时，保留对动作学习仍然有用的预训练VLM表示。

English

Vision-Language-Action (VLA) models widely adopt pretrained Vision-Language Models (VLMs) as policy backbones, yet it remains unclear what kind of pretrained VLM representation is useful as a VLA initialization. In this paper, we study VLA initialization as a controlled representation-design problem along three axes: capability-level embodied VQA supervision, parameter-update strategy, and robot-data pretraining. Our experiments show that the original pretrained VLM representation is a key source of action performance. However, embodied VQA adaptation does not yield uniform gains: its benefit depends on downstream bottlenecks, and gains from different capability domains are not simply additive. For update strategy, LoRA provides a more reliable initialization than Full Finetune, indicating that overly reshaping the pretrained representation can weaken VLA initialization. Robot-data pretraining further improves VLA initialization, with the strongest variant obtained by staged LoRA-based training. Together, these findings suggest that effective VLM-to-VLA adaptation should inject action-relevant embodied and robot-trajectory signals while preserving the pretrained VLM representation that remains useful for action learning.