先看再跳：將樹搜尋蒸餾為凍結VLA模型的動作評估

摘要

視覺-語言-行動（VLA）模型透過大規模預訓練獲得廣泛的具身能力，然而其泛化能力遠比大型語言模型（LLM）與視覺語言模型（VLM）脆弱。常見的補救方式——透過監督式微調或強化學習進行後訓練——雖能提升特定任務的表現，卻會窄化預訓練所帶來的通用能力。我們發現一個關鍵瓶頸：VLA模型的失敗不僅源於動作生成，也源於動作評估。一項診斷性的pass@k研究證實，凍結的VLA在其輸出分佈中已包含具備能力的行為，整體成功率從pass@1的33%提升至pass@32的92%。受此啟發，我們提出SVA（Search, Value, and Act）框架，這是一個簡單的架構，能讓凍結的VLA策略具備長期後果感知能力。SVA首先在模擬中利用蒙地卡羅樹搜索充分探索VLA的輸出分佈，收集多樣化的軌跡並標註經驗回報；接著將這些知識蒸餾成輕量級的Q值模型，用以預測候選動作的預期後果；部署時，凍結的VLA提出多個候選動作，由評估器選出不確定性正則化後Q值最高的動作，過程中無需使用模擬器。透過將動作提案與後果評估分離，SVA保留了VLA主幹模型的泛化能力，同時顯著提升任務成功率。在多個具身基準的實驗中，SVA一致地提升了對未見任務的泛化能力，並展現出強大的測試時擴展行為。引人注目的是，SVA使一個9B參數的VLA模型以低27%的推論延遲，超越27B參數的VLA模型達7個百分點，這表明擴展測試時評估比擴展模型規模更具成本效益。

English

Vision-Language-Action (VLA) models acquire broad embodied capabilities through large-scale pretraining, yet their generalization remains far more fragile than that of LLMs and VLMs. The prevailing remedy, post-training via supervised fine-tuning or reinforcement learning, improves task-specific performance but narrows the generalist capability that makes pretraining valuable. We identify a key bottleneck: VLA failures stem not only from action generation but also from action evaluation. A diagnostic pass@k study confirms that frozen VLAs already contain competent behaviors in their output distribution, with overall success rates rising from 33% at pass@1 to 92% at pass@32. Inspired by this, we propose SVA (Search, Value, and Act), a simple framework that equips frozen VLA policies with long-term consequence awareness. SVA first uses Monte-Carlo tree search in simulation to fully explore the VLA's output distribution and collect diverse trajectories annotated with empirical returns; this knowledge is then distilled into a lightweight Q-value model that predicts the expected consequence of candidate actions; at deployment, the frozen VLA proposes multiple candidates and the evaluator selects the one with the highest uncertainty-regularized Q-value, requiring no simulator access. By decoupling action proposal from consequence evaluation, SVA preserves the generalization capacity of the VLA backbone while substantially improving task success rates. Experiments across embodied benchmarks show that SVA consistently improves generalization on unseen tasks and exhibits strong test-time scaling behavior. Strikingly, SVA enables a 9B VLA to outperform a 27B VLA by 7 points at 27% lower inference latency, suggesting that scaling test-time evaluation is more cost-effective than scaling model size.