VLA真的掌握基础吗？衡量视觉-语言-动作模型中的常识与世界知识保持能力

摘要

具身视觉-语言-动作（VLA）模型通常通过在机器人数据上微调强大的预训练视觉语言模型（VLM）获得，但尚不清楚这些模型在适应后保留了多少常识与事实知识。针对知识敏感任务出现的失败情况具有歧义性，容易混淆知识缺失与底层控制泛化能力不足的问题。我们提出Act2Answer——一种轻量级评估协议，通过要求智能体以动作回答问题，将VLM知识基准适配至VLA评估。每个问题转化为一个简短的桌面场景片段，智能体通过执行单个物体放置动作来选定候选答案，从而得到降低控制混淆因素的、基于动作的成功率。我们构建了一套涵盖多种常识与世界知识类别的测试环境集合，并引入逐层意图探针技术，以定位VLM主干网络和动作头中与答案相关的信息。通过对7个VLA模型和9个VLM基线的大规模研究，我们系统性地对各模型按类别进行排序。结果表明：VLA在简单概念上表现稳健，但在语义更丰富的类别上相较于其源VLM出现了更大的性能差距；VQA联合训练与更好的知识保留能力相关；与答案相关的信号在VLA中间层达到峰值，但在高层逐渐衰减。Act2Answer代码已开源，网址为https://tttonyalpha.github.io/act2answer/。

English

Embodied Vision-Language-Action (VLA) models are typically obtained by fine-tuning powerful pretrained VLMs on robotics data, yet it is unclear how much commonsense and factual knowledge they retain after adaptation. Failures on knowledge-sensitive tasks are ambiguous, conflating missing knowledge with poor generalization of low-level control. We introduce Act2Answer, a lightweight protocol that adapts VLM knowledge benchmarks to VLA evaluation by requiring agents to answer through action. Each question becomes a short tabletop episode where the agent performs a single object-placement action to select among candidate answers, yielding an action-grounded success rate with reduced control confounds. We curate a test suite of such environments across diverse commonsense and world-knowledge categories and introduce layerwise intent probing to localize answer-relevant information across the VLM backbone and action head. In a large-scale study of 7 VLA models and 9 VLM baselines, we systematically rank models across categories, finding that VLAs show solid performance on simple concepts while exhibiting larger gaps on richer semantic categories relative to their source VLMs, that VQA co-training is associated with better knowledge retention, and that answer-relevant signals peak in middle VLA layers but attenuate in upper layers. Act2Answer is available at https://tttonyalpha.github.io/act2answer/.