可驾驶的三维高斯化身

摘要

我们提出了可驾驶的三维高斯化身（D3GA），这是第一个使用高斯斑点渲染的人体三维可控模型。目前逼真的可驾驶化身在训练期间要求准确的三维配准、在测试期间要求密集的输入图像，或者两者兼而有之。基于神经辐射场的模型在远程呈现应用中往往速度过慢。本研究利用最近提出的三维高斯斑点（3DGS）技术以实时帧速率渲染逼真的人体，使用密集校准的多视角视频作为输入。为了变形这些基元，我们摒弃了常用的线性混合蒙皮（LBS）的点变形方法，而是采用了经典的体积变形方法：笼状变形。鉴于它们较小的尺寸，我们使用关节角度和关键点来驱动这些变形，这对于通信应用更为合适。我们在九个主体上进行的实验涵盖了各种体型、服装和动作，结果表明在使用相同的训练和测试数据时，我们的方法比现有技术获得了更高质量的结果。

English

We present Drivable 3D Gaussian Avatars (D3GA), the first 3D controllable model for human bodies rendered with Gaussian splats. Current photorealistic drivable avatars require either accurate 3D registrations during training, dense input images during testing, or both. The ones based on neural radiance fields also tend to be prohibitively slow for telepresence applications. This work uses the recently presented 3D Gaussian Splatting (3DGS) technique to render realistic humans at real-time framerates, using dense calibrated multi-view videos as input. To deform those primitives, we depart from the commonly used point deformation method of linear blend skinning (LBS) and use a classic volumetric deformation method: cage deformations. Given their smaller size, we drive these deformations with joint angles and keypoints, which are more suitable for communication applications. Our experiments on nine subjects with varied body shapes, clothes, and motions obtain higher-quality results than state-of-the-art methods when using the same training and test data.