三思而后行：将树搜索蒸馏为冻结VLA模型的动作评估

摘要

视觉-语言-动作（VLA）模型通过大规模预训练获得了广泛的具身能力，但其泛化能力仍远不及大语言模型和视觉-语言模型那般稳健。当前主流的微调方法（监督微调或强化学习）虽能提升特定任务性能，却也削弱了预训练所赋予的通用能力。我们发现关键瓶颈在于：VLA的失败不仅源于动作生成，更来自动作评估。一项诊断性pass@k研究证实，冻结的VLA模型在其输出分布中已包含合格行为——整体成功率从pass@1的33%提升至pass@32的92%。受此启发，我们提出SVA（搜索、价值与行动）这一简洁框架，为冻结的VLA策略赋予长期后果感知能力。SVA首先在仿真环境中利用蒙特卡洛树搜索充分探索VLA的输出分布，收集带有经验回报标注的多样化轨迹；随后将这些知识蒸馏至轻量级Q值模型，使其能够预测候选动作的预期后果；部署时，冻结的VLA生成多个候选动作，由评估器选择具有最高不确定性正则化Q值的动作，整个过程无需调用仿真器。通过将动作提议与后果评估解耦，SVA既保留了VLA骨干模型的泛化能力，又大幅提升了任务成功率。在具身基准测试上的实验表明，SVA在未见任务上持续提升泛化性能，并展现出强大的测试时扩展行为。尤为显著的是，SVA使9B参数的VLA模型以低27%的推理延迟超越27B模型7个百分点，这说明扩展测试时评估比扩展模型规模更具成本效益。

English

Vision-Language-Action (VLA) models acquire broad embodied capabilities through large-scale pretraining, yet their generalization remains far more fragile than that of LLMs and VLMs. The prevailing remedy, post-training via supervised fine-tuning or reinforcement learning, improves task-specific performance but narrows the generalist capability that makes pretraining valuable. We identify a key bottleneck: VLA failures stem not only from action generation but also from action evaluation. A diagnostic pass@k study confirms that frozen VLAs already contain competent behaviors in their output distribution, with overall success rates rising from 33% at pass@1 to 92% at pass@32. Inspired by this, we propose SVA (Search, Value, and Act), a simple framework that equips frozen VLA policies with long-term consequence awareness. SVA first uses Monte-Carlo tree search in simulation to fully explore the VLA's output distribution and collect diverse trajectories annotated with empirical returns; this knowledge is then distilled into a lightweight Q-value model that predicts the expected consequence of candidate actions; at deployment, the frozen VLA proposes multiple candidates and the evaluator selects the one with the highest uncertainty-regularized Q-value, requiring no simulator access. By decoupling action proposal from consequence evaluation, SVA preserves the generalization capacity of the VLA backbone while substantially improving task success rates. Experiments across embodied benchmarks show that SVA consistently improves generalization on unseen tasks and exhibits strong test-time scaling behavior. Strikingly, SVA enables a 9B VLA to outperform a 27B VLA by 7 points at 27% lower inference latency, suggesting that scaling test-time evaluation is more cost-effective than scaling model size.