GeneOH擴散：朝向可泛化的手-物體互動 通過去噪擴散進行去噪

GeneOH Diffusion: Towards Generalizable Hand-Object Interaction Denoising via Denoising Diffusion

February 22, 2024

作者: Xueyi Liu, Li Yi

cs.AI

摘要

在這項工作中，我們解決了去噪手物互動（HOI）的挑戰性問題。給定一個錯誤的互動序列，目標是對不正確的手部軌跡進行精煉，以消除互動產生的藝術效果，使序列在感知上更加真實。這個挑戰涉及複雜的互動噪聲，包括不自然的手部姿勢和不正確的手物關係，同時需要對新的互動和不同的噪聲模式進行強大的泛化。我們通過一種新穎的方法GeneOH Diffusion 解決了這些挑戰，其中包括兩個關鍵設計：一種名為GeneOH的創新接觸中心的HOI表示形式，以及一種新的具有通用性的去噪方案。接觸中心的表示形式GeneOH 對HOI過程進行了信息化參數化，有助於在各種HOI情境中實現增強泛化。新的去噪方案包括一個經典的去噪模型，用於將從白化噪聲空間中的嘈雜數據樣本投影到乾淨的數據流形，以及一種“通過擴散去噪”的策略，通過首先將具有各種噪聲模式的輸入軌跡擴散以與白化噪聲空間對齊，然後通過經典的去噪器進行清理。在四個具有顯著領域變化的基準測試上進行的大量實驗表明了我們方法的卓越有效性。GeneOH Diffusion 也展示了在各種下游應用中的潛力。項目網站：https://meowuu7.github.io/GeneOH-Diffusion/。

English

In this work, we tackle the challenging problem of denoising hand-object interactions (HOI). Given an erroneous interaction sequence, the objective is to refine the incorrect hand trajectory to remove interaction artifacts for a perceptually realistic sequence. This challenge involves intricate interaction noise, including unnatural hand poses and incorrect hand-object relations, alongside the necessity for robust generalization to new interactions and diverse noise patterns. We tackle those challenges through a novel approach, GeneOH Diffusion, incorporating two key designs: an innovative contact-centric HOI representation named GeneOH and a new domain-generalizable denoising scheme. The contact-centric representation GeneOH informatively parameterizes the HOI process, facilitating enhanced generalization across various HOI scenarios. The new denoising scheme consists of a canonical denoising model trained to project noisy data samples from a whitened noise space to a clean data manifold and a "denoising via diffusion" strategy which can handle input trajectories with various noise patterns by first diffusing them to align with the whitened noise space and cleaning via the canonical denoiser. Extensive experiments on four benchmarks with significant domain variations demonstrate the superior effectiveness of our method. GeneOH Diffusion also shows promise for various downstream applications. Project website: https://meowuu7.github.io/GeneOH-Diffusion/.