TacSL：用于视触觉传感器模拟和学习的库

TacSL: A Library for Visuotactile Sensor Simulation and Learning

August 12, 2024

作者: Iretiayo Akinola, Jie Xu, Jan Carius, Dieter Fox, Yashraj Narang

cs.AI

摘要

对于人类和机器人来说，触觉感知，即触觉传感，在执行接触丰富的操作任务中至关重要。机器人触觉传感面临的三个关键挑战是：1）解释传感器信号，2）在新颖场景中生成传感器信号，3）学习基于传感器的策略。对于视触觉传感器，解释受益于其与视觉传感器（例如RGB摄像头）之间的密切关系。然而，生成仍然困难，因为视触觉传感器通常涉及接触、变形、照明和成像，这些都很昂贵来模拟；反过来，策略学习具有挑战性，因为模拟无法用于大规模数据收集。我们提出了TacSL（taxel），这是一个基于GPU的视触觉传感器模拟和学习库。TacSL可用于在广泛使用的Isaac Gym模拟器中比先前的最先进技术快200倍以上地模拟视触觉图像并提取接触力分布。此外，TacSL提供了一个学习工具包，其中包含多个传感器模型、接触密集型训练环境以及可以促进模拟到真实应用的在线/离线算法。在算法方面，我们引入了一种新颖的在线强化学习算法，称为不对称演员-评论家蒸馏（\sysName），旨在有效且高效地在模拟中学习基于触觉的策略，以便能够转移到真实世界。最后，我们通过评估蒸馏和多模态感知对接触丰富操作任务的益处以及最关键的进行模拟到真实的转移来展示我们的库和算法的效用。补充视频和结果请参见https://iakinola23.github.io/tacsl/。

English

For both humans and robots, the sense of touch, known as tactile sensing, is critical for performing contact-rich manipulation tasks. Three key challenges in robotic tactile sensing are 1) interpreting sensor signals, 2) generating sensor signals in novel scenarios, and 3) learning sensor-based policies. For visuotactile sensors, interpretation has been facilitated by their close relationship with vision sensors (e.g., RGB cameras). However, generation is still difficult, as visuotactile sensors typically involve contact, deformation, illumination, and imaging, all of which are expensive to simulate; in turn, policy learning has been challenging, as simulation cannot be leveraged for large-scale data collection. We present TacSL (taxel), a library for GPU-based visuotactile sensor simulation and learning. TacSL can be used to simulate visuotactile images and extract contact-force distributions over 200times faster than the prior state-of-the-art, all within the widely-used Isaac Gym simulator. Furthermore, TacSL provides a learning toolkit containing multiple sensor models, contact-intensive training environments, and online/offline algorithms that can facilitate policy learning for sim-to-real applications. On the algorithmic side, we introduce a novel online reinforcement-learning algorithm called asymmetric actor-critic distillation (\sysName), designed to effectively and efficiently learn tactile-based policies in simulation that can transfer to the real world. Finally, we demonstrate the utility of our library and algorithms by evaluating the benefits of distillation and multimodal sensing for contact-rich manip ulation tasks, and most critically, performing sim-to-real transfer. Supplementary videos and results are at https://iakinola23.github.io/tacsl/.