无刚性运动转移的时间残差雅可比矩阵

Temporal Residual Jacobians For Rig-free Motion Transfer

July 20, 2024

作者: Sanjeev Muralikrishnan, Niladri Shekhar Dutt, Siddhartha Chaudhuri, Noam Aigerman, Vladimir Kim, Matthew Fisher, Niloy J. Mitra

cs.AI

摘要

我们引入了时间残差雅可比矩阵作为一种新颖的表示形式，以实现基于数据驱动的运动转移。我们的方法不假设访问任何绑定或中间形状关键帧，能够产生几何和时间上一致的运动，并可用于转移长时间序列的运动。我们方法的核心是两个耦合的神经网络，分别预测局部的几何和时间变化，随后将其空间和时间集成，以生成最终的动画网格。这两个网络是联合训练的，相互补充产生空间和时间信号，并直接使用三维位置信息进行监督。在推断过程中，在没有关键帧的情况下，我们的方法实质上解决了一种运动外推问题。我们在各种网格上（合成和扫描形状）测试了我们的设置，以展示其在未见身体形状上生成逼真和自然动画方面优于SoTA替代方案。补充视频和代码可在 https://temporaljacobians.github.io/ 获取。

English

We introduce Temporal Residual Jacobians as a novel representation to enable data-driven motion transfer. Our approach does not assume access to any rigging or intermediate shape keyframes, produces geometrically and temporally consistent motions, and can be used to transfer long motion sequences. Central to our approach are two coupled neural networks that individually predict local geometric and temporal changes that are subsequently integrated, spatially and temporally, to produce the final animated meshes. The two networks are jointly trained, complement each other in producing spatial and temporal signals, and are supervised directly with 3D positional information. During inference, in the absence of keyframes, our method essentially solves a motion extrapolation problem. We test our setup on diverse meshes (synthetic and scanned shapes) to demonstrate its superiority in generating realistic and natural-looking animations on unseen body shapes against SoTA alternatives. Supplemental video and code are available at https://temporaljacobians.github.io/ .