RoboTTT: 机器人策略的上下文扩展

摘要

当前的机器人基础模型通常仅利用单步或短时间跨度的视觉-运动上下文。为此，我们提出机器人测试时训练策略（RoboTTT）——一种机器人模型与训练方案，可将视觉-运动上下文扩展至8000时间步，比现有最先进策略高出三个数量级，且推理延迟不增长。在该上下文长度下，我们解锁了新的机器人能力：从人类视频演示中进行一次性上下文模仿学习、实时策略改进、对扰动的鲁棒性，以及在多阶段长时域任务上更强的性能。我们首次观察到，随着预训练上下文长度增加，闭环性能呈现稳定提升。RoboTTT的核心是将测试时训练集成到视觉-语言-动作策略等机器人基础模型中，生成一种序列模型，其循环状态由快速权重构成——这些参数在训练和推理期间通过梯度下降更新，将历史信息压缩至权重空间，并检索上下文信息以实现长上下文条件控制。为扩展训练上下文长度，该方案结合了序列动作强制与截断时间反向传播。在具有挑战性的真实机器人操作任务上，RoboTTT相较于单步上下文基线整体性能提升87%，并完整完成了一项耗时五分钟、包含十个阶段的装配任务——这是所有基线方法均未能实现的。采用8000时间步上下文训练的RoboTTT，比同一模型在1000时间步下预训练的性能高出62%，表明上下文长度可作为机器人基础模型的新缩放维度。视频请见https://research.nvidia.com/labs/gear/robottt/。

English

Recent robot foundation models operate with single-step or short-history visuomotor context. We introduce Test-Time-Training Robot Policies (RoboTTT), a robot model and training recipe that scale visuomotor context to 8K timesteps, three orders of magnitude beyond state-of-the-art policies, without growing inference latency. At this context length, we unlock new robot capabilities: one-shot in-context imitation from human video demonstrations, on-the-fly policy improvement, robustness to perturbations, and stronger performance on multi-stage, long-horizon tasks. We also observe, for the first time, steady gains in closed-loop performance as pretraining context length scales. At its core, RoboTTT integrates Test-Time Training into robot foundation models such as Vision-Language-Action policies, yielding a sequence model whose recurrent state consists of fast weights, parameters updated by gradient descent during both training and inference, compressing histories into weight space and retrieving contextual information for long-context conditioning. To scale training context length, the recipe combines sequence action forcing with truncated backpropagation through time. On challenging real-robot manipulation tasks, RoboTTT improves overall performance by 87% over the single-step context baseline and fully completes a five-minute, ten-stage assembly task, which no baseline ever does. RoboTTT trained with 8K-timestep context outperforms the same model pretrained with 1K timesteps by 62%, suggesting context length as a new scaling axis for robot foundation models. Videos are available at https://research.nvidia.com/labs/gear/robottt/