轉換作為橋接動作：將操作技能從人類轉移至機器人

摘要

我們研究是否能從人類動作中學習新穎的操作技能，並將其遷移至配備平行夾爪的雙臂機器人。人類動作資料成本低廉、豐富且多樣，是擴展機器人學習最具前景的資源之一。然而，將技能從人類遷移至機器人仍具挑戰：多數先前研究將人類視為另一種雙臂六自由度本體，其中手部姿態估計充滿雜訊，且人類手指的接觸模式與平行夾爪存在根本差異。我們認為，從人類資料學習包含旋轉的動作訊號因此並非最佳解，並提出一種橋接動作表徵：初始頭戴攝影機座標系內的相對手腕平移，這是人類與機器人共有的動作空間。為處理不同本體中可能缺失特定動作組件的問題，我們建構了一個類似π_0的視覺-語言-動作模型，採用交錯式動作令牌與注意力遮罩。在一系列新穎的雙臂操作任務中，我們的橋接動作能將人類操作知識遷移至機器人，效果遠優於含雜訊的六自由度人類動作，並能隨人類資料量擴展。

English

We study whether we can learn novel manipulation skills from human actions to a bi-manual robot with parallel grippers. Human action data is cheap, abundant, and diverse, making it one of the most promising resources for scaling up robot learning. Yet transferring skills from humans to robots remains hard: most prior work treats humans as just another bi-manual 6DoF embodiment, where hand-pose estimates are noisy and the contact patterns of human fingers differ fundamentally from those of a parallel gripper. We argue that learning rotation-inclusive action signals from human data is therefore sub-optimal, and instead propose a bridging action representation: the relative wrist translation within the initial head-camera frame, an action space shared by humans and robots. To handle the potential absence of certain action components in different embodiments, we build a π_0-like vision-language-action model with interleaved action tokens and attention masking. On a suite of novel bi-manual manipulation tasks, our bridging action transfers human manipulation knowledge to robots far more effectively than noisy 6DoF human actions and scales with the amount of human data.