PolicyTrim：提升视觉-语言-动作模型的内在策略效率

摘要

視覺-語言-動作（VLA）模型為機器人操作提供了統一的範式，然而其在現實世界中的部署常因執行效率而受限。現有研究主要聚焦於以計算為中心的效率，以降低每次推論的延遲，但這些模型內在的策略效率仍未被充分探討。策略效率根本上受兩個因素影響：預測動作塊的有效可執行長度，以及完成任務所需的總物理步驟數。這兩個因素共同決定了執行期間前向推論調用的總次數。我們觀察到，當前的VLA策略在規劃可靠性與動作冗餘方面存在問題，其動作塊尾部的預測品質嚴重下降，且傾向於產生不必要的冗餘物理步驟。為解決此問題，我們提出PolicyTrim，這是一個基於強化學習的後訓練框架，能延長可靠動作塊長度並減少冗餘物理步驟。在可靠區塊延伸方面，我們採用動態探索策略，明確獎勵成功完成更長可執行長度的行為，逐步將可信預測範圍推向其實驗極限。在步驟效率方面，我們設計了冗餘感知獎勵，直接獎勵以更少步驟成功完成任務的行為，同時懲罰不可複現的捷徑，從而有效消除冗餘物理動作。在三項基準測試及三種VLA模型上的大量實驗表明，PolicyTrim將動作塊利用率提升3倍，並減少51.4%的物理執行步驟。最終，我們的框架在不影響任務成功率的條件下，實現了高達5.83倍的端到端部署加速。

English

Vision-Language-Action (VLA) models provide a unified paradigm for robotic manipulation, yet their real-world deployment is often bottlenecked by execution efficiency. While existing efforts predominantly focus on compute-centric efficiency to reduce per-step inference latency, the intrinsic policy efficiency of these models remains largely unexplored. Policy efficiency is fundamentally affected by two factors, namely the effective executable length of predicted action chunks and the total physical steps required to complete a task. These two factors jointly determine the total number of forward inference calls during execution. We observe that current VLA policies struggle with planning unreliability and action redundancy, suffering from severe prediction degradation at the tail of action chunks and tending to generate unnecessarily redundant physical steps. To address this, we propose PolicyTrim, a reinforcement learning-based post-training framework that extends the reliable action chunk length and reduces redundant physical steps. For reliable chunk extension, we employ a dynamic exploration strategy that explicitly rewards the successful completion of longer executable lengths, progressively pushing the trustworthy prediction horizon to its empirical limit. For step efficiency, we design a redundancy-aware reward that directly favors successful task completions with fewer steps while penalizing unreproducible shortcuts, effectively eliminating redundant physical actions. Extensive experiments across three benchmarks and three VLA models demonstrate that PolicyTrim improves action chunk utilization by 3times and reduces physical execution steps by 51.4\%. Ultimately, our framework delivers up to a 5.83times end-to-end deployment speedup without compromising task success rates.