实时模拟化身的永续人形控制

摘要

我们提出了一种基于物理的人形控制器，能够在存在嘈杂输入（例如来自视频的姿势估计或语言生成的姿势）和意外摔倒的情况下实现高保真度的动作模仿和容错行为。我们的控制器能够扩展到学习一万个动作片段，而无需使用任何外部稳定力，并学会自然地从失败状态中恢复。在给定参考动作的情况下，我们的控制器可以持续控制模拟化身，而无需重置。在其核心，我们提出了渐进式乘性控制策略（PMCP），动态分配新的网络容量来学习越来越困难的动作序列。PMCP允许有效地扩展学习大规模动作数据库和添加新任务，例如从失败状态中恢复，而不会发生灾难性遗忘。我们通过在实时多人化身使用案例中使用它来模仿来自基于视频的姿势估计器和基于语言的动作生成器的嘈杂姿势，展示了我们控制器的有效性。

English

We present a physics-based humanoid controller that achieves high-fidelity motion imitation and fault-tolerant behavior in the presence of noisy input (e.g. pose estimates from video or generated from language) and unexpected falls. Our controller scales up to learning ten thousand motion clips without using any external stabilizing forces and learns to naturally recover from fail-state. Given reference motion, our controller can perpetually control simulated avatars without requiring resets. At its core, we propose the progressive multiplicative control policy (PMCP), which dynamically allocates new network capacity to learn harder and harder motion sequences. PMCP allows efficient scaling for learning from large-scale motion databases and adding new tasks, such as fail-state recovery, without catastrophic forgetting. We demonstrate the effectiveness of our controller by using it to imitate noisy poses from video-based pose estimators and language-based motion generators in a live and real-time multi-person avatar use case.

实时模拟化身的永续人形控制

Perpetual Humanoid Control for Real-time Simulated Avatars

摘要

Support