UniT：統一觸覺表示法用於機器人學習

UniT: Unified Tactile Representation for Robot Learning

August 12, 2024

作者: Zhengtong Xu, Raghava Uppuluri, Xinwei Zhang, Cael Fitch, Philip Glen Crandall, Wan Shou, Dongyi Wang, Yu She

cs.AI

摘要

UniT是一種新穎的觸覺表示學習方法，利用VQVAE來學習緊湊的潛在空間並作為觸覺表示。它使用從單一簡單物體獲得的觸覺圖像來訓練具有可轉移性和泛化性的表示。這種觸覺表示可以零樣本轉移到各種下游任務，包括感知任務和操作策略學習。我們在手中3D姿勢估計任務上的基準測試顯示，UniT優於現有的視覺和觸覺表示學習方法。此外，UniT在政策學習方面的有效性已在涉及多樣操縱物體和複雜機器人-物體-環境交互作用的三個現實世界任務中得到證明。通過大量實驗，UniT被證明是一種易於訓練、即插即用，但廣泛有效的觸覺表示學習方法。有關更多詳細信息，請參閱我們的開源存儲庫https://github.com/ZhengtongXu/UniT和項目網站https://zhengtongxu.github.io/unifiedtactile.github.io/。

English

UniT is a novel approach to tactile representation learning, using VQVAE to learn a compact latent space and serve as the tactile representation. It uses tactile images obtained from a single simple object to train the representation with transferability and generalizability. This tactile representation can be zero-shot transferred to various downstream tasks, including perception tasks and manipulation policy learning. Our benchmarking on an in-hand 3D pose estimation task shows that UniT outperforms existing visual and tactile representation learning methods. Additionally, UniT's effectiveness in policy learning is demonstrated across three real-world tasks involving diverse manipulated objects and complex robot-object-environment interactions. Through extensive experimentation, UniT is shown to be a simple-to-train, plug-and-play, yet widely effective method for tactile representation learning. For more details, please refer to our open-source repository https://github.com/ZhengtongXu/UniT and the project website https://zhengtongxu.github.io/unifiedtactile.github.io/.