TRANSIC：通过从在线校正中学习实现从模拟到真实的策略迁移

TRANSIC: Sim-to-Real Policy Transfer by Learning from Online Correction

May 16, 2024

作者: Yunfan Jiang, Chen Wang, Ruohan Zhang, Jiajun Wu, Li Fei-Fei

cs.AI

摘要

在模拟环境中学习并将学到的策略转移到现实世界，有潜力实现通用型机器人。这种方法的关键挑战是解决模拟到现实（sim-to-real）之间的差距。先前的方法通常需要先验的领域特定知识。我们认为获得这种知识的一种直接方式是请人类观察和协助机器人在现实世界执行策略。然后机器人可以从人类那里学习，以消除各种模拟到现实的差距。我们提出了TRANSIC，这是一种基于人在回路中的数据驱动方法，以实现成功的模拟到现实转移。TRANSIC允许人类通过干预和在线纠正来增强模拟策略，从而全面地克服各种未建模的模拟到现实差距。残余策略可以从人类的纠正中学习，并与模拟策略集成以进行自主执行。我们展示了我们的方法可以在复杂和接触丰富的操纵任务（如家具组装）中实现成功的模拟到现实转移。通过在模拟中学习的策略与人类的策略的协同集成，TRANSIC作为一种全面解决各种常常共存的模拟到现实差距的方法是有效的。它展现出随着人类努力而扩展的吸引人的特性。视频和代码可在https://transic-robot.github.io/获得。

English

Learning in simulation and transferring the learned policy to the real world has the potential to enable generalist robots. The key challenge of this approach is to address simulation-to-reality (sim-to-real) gaps. Previous methods often require domain-specific knowledge a priori. We argue that a straightforward way to obtain such knowledge is by asking humans to observe and assist robot policy execution in the real world. The robots can then learn from humans to close various sim-to-real gaps. We propose TRANSIC, a data-driven approach to enable successful sim-to-real transfer based on a human-in-the-loop framework. TRANSIC allows humans to augment simulation policies to overcome various unmodeled sim-to-real gaps holistically through intervention and online correction. Residual policies can be learned from human corrections and integrated with simulation policies for autonomous execution. We show that our approach can achieve successful sim-to-real transfer in complex and contact-rich manipulation tasks such as furniture assembly. Through synergistic integration of policies learned in simulation and from humans, TRANSIC is effective as a holistic approach to addressing various, often coexisting sim-to-real gaps. It displays attractive properties such as scaling with human effort. Videos and code are available at https://transic-robot.github.io/