DynaMo：面向视觉-运动控制的领域内动态预训练

DynaMo: In-Domain Dynamics Pretraining for Visuo-Motor Control

September 18, 2024

作者: Zichen Jeff Cui, Hengkai Pan, Aadhithya Iyer, Siddhant Haldar, Lerrel Pinto

cs.AI

摘要

模仿学习已被证明是训练复杂视觉动作策略的强大工具。然而，目前的方法通常需要数百到数千个专家演示来处理高维视觉观测。造成这种数据效率低的一个关键原因是，视觉表示主要是预训练在域外数据上或直接通过行为克隆目标进行训练。在这项工作中，我们提出了DynaMo，一种新的域内自监督学习视觉表示方法。给定一组专家演示，我们共同学习一个潜在的逆动力学模型和一个正向动力学模型，通过一系列图像嵌入来预测潜在空间中的下一帧，无需增强、对比采样或访问地面真实动作。重要的是，DynaMo 不需要任何域外数据，如互联网数据集或跨体数据集。在六个模拟和真实环境套件中，我们展示了使用DynaMo学习的表示显著改善了先前自监督学习目标和预训练表示的下游模仿学习性能。使用DynaMo获得的收益在行为变换器、扩散策略、MLP 和最近邻等策略类别中都适用。最后，我们对DynaMo的关键组件进行了消融实验，并测量其对下游策略性能的影响。机器人视频最佳观看网址为 https://dynamo-ssl.github.io

English

Imitation learning has proven to be a powerful tool for training complex visuomotor policies. However, current methods often require hundreds to thousands of expert demonstrations to handle high-dimensional visual observations. A key reason for this poor data efficiency is that visual representations are predominantly either pretrained on out-of-domain data or trained directly through a behavior cloning objective. In this work, we present DynaMo, a new in-domain, self-supervised method for learning visual representations. Given a set of expert demonstrations, we jointly learn a latent inverse dynamics model and a forward dynamics model over a sequence of image embeddings, predicting the next frame in latent space, without augmentations, contrastive sampling, or access to ground truth actions. Importantly, DynaMo does not require any out-of-domain data such as Internet datasets or cross-embodied datasets. On a suite of six simulated and real environments, we show that representations learned with DynaMo significantly improve downstream imitation learning performance over prior self-supervised learning objectives, and pretrained representations. Gains from using DynaMo hold across policy classes such as Behavior Transformer, Diffusion Policy, MLP, and nearest neighbors. Finally, we ablate over key components of DynaMo and measure its impact on downstream policy performance. Robot videos are best viewed at https://dynamo-ssl.github.io