GeneOH扩散：通向可泛化的手-物体交互 通过去噪扩散进行去噪

GeneOH Diffusion: Towards Generalizable Hand-Object Interaction Denoising via Denoising Diffusion

February 22, 2024

作者: Xueyi Liu, Li Yi

cs.AI

摘要

在这项工作中，我们解决了去噪手-物体交互（HOI）的挑战性问题。给定一个错误的交互序列，目标是优化不正确的手部轨迹，消除交互伪影，以获得感知上真实的序列。这一挑战涉及复杂的交互噪声，包括不自然的手部姿势和不正确的手-物体关系，同时需要对新的交互和不同的噪声模式具有强大的泛化能力。我们通过一种新颖的方法GeneOH Diffusion 来解决这些挑战，其中包括两个关键设计：一种名为GeneOH的创新接触中心的HOI表示法和一种新的领域通用的去噪方案。接触中心的表示法GeneOH 信息化地参数化了HOI过程，促进了在各种HOI情景中的增强泛化。新的去噪方案包括一个经典去噪模型，经过训练，能够将来自白化噪声空间的嘈杂数据样本投影到干净的数据流形上，以及一种“通过扩散去噪”的策略，通过首先将输入轨迹扩散到与白化噪声空间对齐，然后通过经典去噪器进行清理，从而能够处理具有各种噪声模式的输入轨迹。在四个具有显著领域变化的基准测试上进行的大量实验表明了我们方法的卓越有效性。GeneOH Diffusion 还展示了在各种下游应用中的潜力。项目网站：https://meowuu7.github.io/GeneOH-Diffusion/。

English

In this work, we tackle the challenging problem of denoising hand-object interactions (HOI). Given an erroneous interaction sequence, the objective is to refine the incorrect hand trajectory to remove interaction artifacts for a perceptually realistic sequence. This challenge involves intricate interaction noise, including unnatural hand poses and incorrect hand-object relations, alongside the necessity for robust generalization to new interactions and diverse noise patterns. We tackle those challenges through a novel approach, GeneOH Diffusion, incorporating two key designs: an innovative contact-centric HOI representation named GeneOH and a new domain-generalizable denoising scheme. The contact-centric representation GeneOH informatively parameterizes the HOI process, facilitating enhanced generalization across various HOI scenarios. The new denoising scheme consists of a canonical denoising model trained to project noisy data samples from a whitened noise space to a clean data manifold and a "denoising via diffusion" strategy which can handle input trajectories with various noise patterns by first diffusing them to align with the whitened noise space and cleaning via the canonical denoiser. Extensive experiments on four benchmarks with significant domain variations demonstrate the superior effectiveness of our method. GeneOH Diffusion also shows promise for various downstream applications. Project website: https://meowuu7.github.io/GeneOH-Diffusion/.