TokenHSI：通過任務令牌化實現物理人體-場景交互的統一合成

摘要

合成多樣且物理合理的人類-場景交互（HSI）對於計算機動畫和具身人工智慧至關重要。儘管取得了令人鼓舞的進展，但現有方法主要集中於開發獨立的控制器，每個控制器專注於特定的交互任務。這極大地限制了處理多種需要整合多種技能的複雜HSI任務的能力，例如在攜帶物體時坐下。為解決這一問題，我們提出了TokenHSI，這是一種基於Transformer的單一統一策略，能夠實現多技能統一和靈活適應。關鍵見解是將人形本體感覺建模為一個獨立的共享令牌，並通過掩碼機制將其與不同的任務令牌結合。這種統一策略促進了跨技能的有效知識共享，從而支持多任務訓練。此外，我們的策略架構支持可變長度輸入，使學習到的技能能夠靈活適應新場景。通過訓練額外的任務令牌生成器，我們不僅可以修改交互目標的幾何形狀，還能協調多種技能以應對複雜任務。實驗表明，我們的方法在各種HSI任務中顯著提升了多功能性、適應性和可擴展性。網站：https://liangpan99.github.io/TokenHSI/

English

Synthesizing diverse and physically plausible Human-Scene Interactions (HSI) is pivotal for both computer animation and embodied AI. Despite encouraging progress, current methods mainly focus on developing separate controllers, each specialized for a specific interaction task. This significantly hinders the ability to tackle a wide variety of challenging HSI tasks that require the integration of multiple skills, e.g., sitting down while carrying an object. To address this issue, we present TokenHSI, a single, unified transformer-based policy capable of multi-skill unification and flexible adaptation. The key insight is to model the humanoid proprioception as a separate shared token and combine it with distinct task tokens via a masking mechanism. Such a unified policy enables effective knowledge sharing across skills, thereby facilitating the multi-task training. Moreover, our policy architecture supports variable length inputs, enabling flexible adaptation of learned skills to new scenarios. By training additional task tokenizers, we can not only modify the geometries of interaction targets but also coordinate multiple skills to address complex tasks. The experiments demonstrate that our approach can significantly improve versatility, adaptability, and extensibility in various HSI tasks. Website: https://liangpan99.github.io/TokenHSI/

TokenHSI：通過任務令牌化實現物理人體-場景交互的統一合成

TokenHSI: Unified Synthesis of Physical Human-Scene Interactions through Task Tokenization

摘要

Summary

Support

Support