RoboTAP：用於少樣本視覺模仿的任意點追踪

摘要

為了讓機器人在實驗室以外的環境和特定工廠中發揮作用，我們需要一種能夠快速教導它們新的有用行為的方法。目前的方法要麼缺乏足夠的通用性，無法在沒有特定任務工程的情況下引入新任務，要麼缺乏足夠的數據效率，無法在實際應用中使用的時間內完成這一過程。在這項研究中，我們探索了密集跟踪作為一種表徵工具，以實現更快速、更通用的示範學習。我們的方法利用“跟蹤任意點”（TAP）模型來分離示範中的相關運動，並對低層控制器進行參數化，以在場景配置變化時重現這種運動。我們展示了這導致了能夠解決複雜的物體排列任務，如形狀匹配、堆疊，甚至全程跟隨任務，如應用膠水並將物體粘合在一起的強健機器人策略，所有這些都是通過幾分鐘內收集的示範完成的。

English

For robots to be useful outside labs and specialized factories we need a way to teach them new useful behaviors quickly. Current approaches lack either the generality to onboard new tasks without task-specific engineering, or else lack the data-efficiency to do so in an amount of time that enables practical use. In this work we explore dense tracking as a representational vehicle to allow faster and more general learning from demonstration. Our approach utilizes Track-Any-Point (TAP) models to isolate the relevant motion in a demonstration, and parameterize a low-level controller to reproduce this motion across changes in the scene configuration. We show this results in robust robot policies that can solve complex object-arrangement tasks such as shape-matching, stacking, and even full path-following tasks such as applying glue and sticking objects together, all from demonstrations that can be collected in minutes.

RoboTAP：用於少樣本視覺模仿的任意點追踪

RoboTAP: Tracking Arbitrary Points for Few-Shot Visual Imitation

摘要

Support