VLA真的懂基礎知識嗎？量測視覺-語言-行動模型中的常識與世界知識保留

摘要

具身视觉-语言-行动（VLA）模型通常通过将强大的预训练视觉语言模型（VLM）在机器人数据上进行微调而获得，但在适应后，这些模型保留了多少常识性和事实性知识仍不清楚。在知识敏感型任务上的失败具有模糊性，常将知识缺失与低层级控制泛化不佳混为一谈。我们提出了Act2Answer，这是一种轻量级协议，通过要求智能体以行动作答，将VLM知识基准调整为VLA评估。每个问题构成一段简短的桌面上场景，其中智能体执行一次单一的物体放置动作以选择候选答案，从而得出以行动为基础的成功率，并减少了控制层面的干扰。我们针对各种常识和世界知识类别，策划了一套此类测试环境，并引入分层意图探测技术，以定位VLM骨干网络和行动头部中与答案相关的信息。在一项涵盖7个VLA模型和9个VLM基线的大规模研究中，我们系统性地对各模型在不同类别上进行排名，发现VLA在简单概念上表现稳健，但在更丰富的语义类别上，相比其源VLM存在较大差距；视觉问答（VQA）联合训练与更好的知识保留相关；且与答案相关的信号在VLA中间层达到峰值，但在上层减弱。Act2Answer可在https://tttonyalpha.github.io/act2answer/获取。

English

Embodied Vision-Language-Action (VLA) models are typically obtained by fine-tuning powerful pretrained VLMs on robotics data, yet it is unclear how much commonsense and factual knowledge they retain after adaptation. Failures on knowledge-sensitive tasks are ambiguous, conflating missing knowledge with poor generalization of low-level control. We introduce Act2Answer, a lightweight protocol that adapts VLM knowledge benchmarks to VLA evaluation by requiring agents to answer through action. Each question becomes a short tabletop episode where the agent performs a single object-placement action to select among candidate answers, yielding an action-grounded success rate with reduced control confounds. We curate a test suite of such environments across diverse commonsense and world-knowledge categories and introduce layerwise intent probing to localize answer-relevant information across the VLM backbone and action head. In a large-scale study of 7 VLA models and 9 VLM baselines, we systematically rank models across categories, finding that VLAs show solid performance on simple concepts while exhibiting larger gaps on richer semantic categories relative to their source VLMs, that VQA co-training is associated with better knowledge retention, and that answer-relevant signals peak in middle VLA layers but attenuate in upper layers. Act2Answer is available at https://tttonyalpha.github.io/act2answer/.