NIFTY：用於引導人類動作合成的神經物體交互作用場

NIFTY: Neural Object Interaction Fields for Guided Human Motion Synthesis

July 14, 2023

作者: Nilesh Kulkarni, Davis Rempe, Kyle Genova, Abhijit Kundu, Justin Johnson, David Fouhey, Leonidas Guibas

cs.AI

摘要

我們解決了在場景中生成與物體互動的逼真3D人體動作的問題。我們的關鍵想法是創建一個附加到特定物體的神經互動場，該場輸出給定人體姿勢的情況下到有效互動流形的距離。這種互動場引導對以物體為條件的人體運動擴散模型的採樣，從而鼓勵合理的接觸和功能語義。為了支持與稀缺數據的互動，我們提出了一個自動合成數據管道。為此，我們使用從有限運動捕捉數據中提取的特定互動錨點姿勢來種子化預先訓練的運動模型，該模型對人體運動基礎有先驗知識。使用我們在生成的合成數據上訓練的引導擴散模型，我們綜合了坐姿和舉起多個物體的逼真動作，勝過其他方法，無論是在動作質量還是成功動作完成方面。我們將我們的框架稱為NIFTY：用於軌跡合成的神經互動場。

English

We address the problem of generating realistic 3D motions of humans interacting with objects in a scene. Our key idea is to create a neural interaction field attached to a specific object, which outputs the distance to the valid interaction manifold given a human pose as input. This interaction field guides the sampling of an object-conditioned human motion diffusion model, so as to encourage plausible contacts and affordance semantics. To support interactions with scarcely available data, we propose an automated synthetic data pipeline. For this, we seed a pre-trained motion model, which has priors for the basics of human movement, with interaction-specific anchor poses extracted from limited motion capture data. Using our guided diffusion model trained on generated synthetic data, we synthesize realistic motions for sitting and lifting with several objects, outperforming alternative approaches in terms of motion quality and successful action completion. We call our framework NIFTY: Neural Interaction Fields for Trajectory sYnthesis.