πR²: 反應式即時流量策略

摘要

通用操作策略越来越多地采用基于大型预训练骨干网络的动作分块流策略。这类动作分块以开环方式运行，导致策略无法对执行过程中的感官输入做出响应，牺牲了反应能力。更频繁地重新规划可以恢复反应性，但感知到动作的流水线（大型骨干网络加上多个去噪步骤）过于缓慢：这种延迟阻碍了频繁重新规划，并使得已执行的动作变得过时，导致此类策略难以适用于动态的闭环控制。我们提出πR²，它使这些策略具备反应性和实时性，同时保留大型骨干网络、表达性多模态策略以及多动作预测能力。基于扩散强制的每位置噪声调度，πR²贡献了两项核心思想。第一，它将条件输入拆分为快速通道（本体感知，每拍刷新）和异步更新的慢速通道（视觉语言特征），使得策略能在动作分块内对本体感知做出反应，同时容忍陈旧的视觉信息。第二，一种延迟自适应流调度将正在执行中的动作视为修复条件，并在每次调用时通过单步去噪生成动作，使单一训练模型能适应不同硬件延迟。πR²只需对现有架构进行极小修改，即可从预训练策略微调得到：将其应用于GR00T-N1.7，在真实xArm6+XHand平台上，其闭环重新规划速度约为基础策略的4倍（在A5000 GPU上约25Hz），每40ms基于全新观测执行动作。在模拟和真实世界操作任务中，πR²相比最强基线，在模拟环境中成功率最高提升23%，在真实环境中提升30%。项目页面：https://pi-r2-flow.github.io/

English

Generalist manipulation policies increasingly take the form of action-chunking flow policies built on large pretrained backbones. Such chunks run open-loop, so the policy cannot react to sensory input arriving mid-execution, sacrificing reactivity. Replanning more often would restore it, but the perception-to-action pipeline (a large backbone plus multiple denoising steps) is too slow: this latency forbids frequent replanning and leaves committed actions stale, making such policies ill-suited for dynamic, closed-loop control. We present πR^2, which makes these policies reactive and real-time while retaining large backbones, expressive multi-modal policies, and multi-action prediction. Built on the per-position noise schedule of diffusion forcing, πR^2 contributes two ideas. First, it splits conditioning into a fast channel (proprioception, fresh every tick) and an asynchronously updated slow channel (vision-language features), so the policy reacts to proprioception within a chunk while tolerating stale vision. Second, a latency-adaptive flow schedule treats in-flight actions as inpainting conditioning and emits actions in one denoising step per call, letting one trained model adapt to varying hardware latency. Requiring minimal modification to existing architectures, πR^2 can be finetuned from a pretrained policy: applied to GR00T-N1.7 on a real xArm6+XHand platform, it replans closed-loop roughly 4times faster than the base policy (~25Hz on an A5000 GPU), acting on a fresh observation every 40ms. Across simulation and real-world manipulation tasks, πR^2 improves the success rate by up to 23% in simulation and 30% in the real world over the strongest baseline. Project page: https://pi-r2-flow.github.io/