DragAPart：为关节对象学习部件级运动先验

DragAPart: Learning a Part-Level Motion Prior for Articulated Objects

March 22, 2024

作者: Ruining Li, Chuanxia Zheng, Christian Rupprecht, Andrea Vedaldi

cs.AI

摘要

我们介绍了DragAPart，这是一种方法，给定一幅图像和一组拖动作为输入，可以生成一个新的图像，展示相同物体的新状态，与拖动的动作相兼容。与之前侧重于重新定位物体的作品不同，DragAPart预测部分级别的交互，比如打开和关闭抽屉。我们将这个问题作为学习通用运动模型的代理，不限于特定的运动结构或物体类别。为此，我们从一个预先训练好的图像生成器开始，并在一个新的合成数据集Drag-a-Move上进行微调，该数据集由我们引入。结合一种新的拖动编码和数据集随机化，新模型很好地推广到真实图像和不同类别。与之前的运动控制生成器相比，我们展示了更好的部分级别运动理解能力。

English

We introduce DragAPart, a method that, given an image and a set of drags as input, can generate a new image of the same object in a new state, compatible with the action of the drags. Differently from prior works that focused on repositioning objects, DragAPart predicts part-level interactions, such as opening and closing a drawer. We study this problem as a proxy for learning a generalist motion model, not restricted to a specific kinematic structure or object category. To this end, we start from a pre-trained image generator and fine-tune it on a new synthetic dataset, Drag-a-Move, which we introduce. Combined with a new encoding for the drags and dataset randomization, the new model generalizes well to real images and different categories. Compared to prior motion-controlled generators, we demonstrate much better part-level motion understanding.