身體變形器：利用機器人具體化進行政策學習

Body Transformer: Leveraging Robot Embodiment for Policy Learning

August 12, 2024

作者: Carmelo Sferrazza, Dun-Ming Huang, Fangchen Liu, Jongmin Lee, Pieter Abbeel

cs.AI

摘要

近年來，Transformer 結構已成為自然語言處理和計算機視覺應用的機器學習算法的事實標準。儘管在機器人學習的背景下成功部署該結構的證據引人注目，我們主張純粹的 Transformer 並未充分利用機器人學習問題的結構。因此，我們提出 Body Transformer（BoT），一種利用機器人實體的架構，通過提供引導學習過程的歸納偏差。我們將機器人身體表示為感測器和執行器的圖形，並依賴遮罩注意力在整個結構中池化信息。由此產生的結構在任務完成、擴展性和計算效率方面優於純粹的 Transformer，以及傳統的多層感知器，無論是表示模仿還是強化學習策略。包括開源代碼在內的其他資料可在 https://sferrazza.cc/bot_site 找到。

English

In recent years, the transformer architecture has become the de facto standard for machine learning algorithms applied to natural language processing and computer vision. Despite notable evidence of successful deployment of this architecture in the context of robot learning, we claim that vanilla transformers do not fully exploit the structure of the robot learning problem. Therefore, we propose Body Transformer (BoT), an architecture that leverages the robot embodiment by providing an inductive bias that guides the learning process. We represent the robot body as a graph of sensors and actuators, and rely on masked attention to pool information throughout the architecture. The resulting architecture outperforms the vanilla transformer, as well as the classical multilayer perceptron, in terms of task completion, scaling properties, and computational efficiency when representing either imitation or reinforcement learning policies. Additional material including the open-source code is available at https://sferrazza.cc/bot_site.