DragAPart：為關節物體學習部分級別運動先驗

DragAPart: Learning a Part-Level Motion Prior for Articulated Objects

March 22, 2024

作者: Ruining Li, Chuanxia Zheng, Christian Rupprecht, Andrea Vedaldi

cs.AI

摘要

我們介紹了DragAPart，一種方法，給定一幅圖像和一組拖動作為輸入，可以生成一幅新的圖像，展現物體在新狀態下的動作，與拖動的動作相容。與先前專注於重新定位物體的作品不同，DragAPart 預測部分級別的互動，例如打開和關閉抽屜。我們將這個問題作為學習通用運動模型的代理，不限於特定的運動結構或物體類別。為此，我們從一個預先訓練的圖像生成器開始，並在一個新的合成數據集 Drag-a-Move 上進行微調，我們也介紹了這個數據集。結合一種新的拖動編碼和數據集隨機化，新模型對真實圖像和不同類別有很好的泛化能力。與先前的運動控制生成器相比，我們展示了更好的部分級別運動理解。

English

We introduce DragAPart, a method that, given an image and a set of drags as input, can generate a new image of the same object in a new state, compatible with the action of the drags. Differently from prior works that focused on repositioning objects, DragAPart predicts part-level interactions, such as opening and closing a drawer. We study this problem as a proxy for learning a generalist motion model, not restricted to a specific kinematic structure or object category. To this end, we start from a pre-trained image generator and fine-tune it on a new synthetic dataset, Drag-a-Move, which we introduce. Combined with a new encoding for the drags and dataset randomization, the new model generalizes well to real images and different categories. Compared to prior motion-controlled generators, we demonstrate much better part-level motion understanding.