BEDLAM：展示详细栩栩如生动态的人体合成数据集

BEDLAM: A Synthetic Dataset of Bodies Exhibiting Detailed Lifelike Animated Motion

June 29, 2023

作者: Michael J. Black, Priyanka Patel, Joachim Tesch, Jinlong Yang

cs.AI

摘要

我们首次展示，仅基于合成数据训练的神经网络在从真实图像中估计3D人体姿势和形状（HPS）问题上实现了最先进的准确性。先前的合成数据集规模较小，不够真实，或缺乏真实服装。实现足够的真实感并非易事，我们展示了如何在运动中实现全身的真实感。具体来说，我们的BEDLAM数据集包含单目RGB视频，其中包含以SMPL-X格式的地面真实3D人体。它包括多样的体型、动作、肤色、头发和服装。服装是通过商业服装物理模拟在移动的人体上逼真模拟的。我们在真实场景中呈现不同数量的人员，具有多样化的光照和摄像机运动。然后，我们使用BEDLAM训练各种HPS回归器，并在真实图像基准上实现最先进的准确性，尽管是使用合成数据进行训练。我们利用BEDLAM来深入了解哪些模型设计选择对准确性至关重要。通过良好的合成训练数据，我们发现像HMR这样的基本方法接近当前SOTA方法（CLIFF）的准确性。BEDLAM对各种任务都很有用，所有图像、地面真实人体、3D服装、支持代码等均可供研究目的使用。此外，我们提供有关我们合成数据生成流程的详细信息，使其他人能够生成自己的数据集。请查看项目页面：https://bedlam.is.tue.mpg.de/。

English

We show, for the first time, that neural networks trained only on synthetic data achieve state-of-the-art accuracy on the problem of 3D human pose and shape (HPS) estimation from real images. Previous synthetic datasets have been small, unrealistic, or lacked realistic clothing. Achieving sufficient realism is non-trivial and we show how to do this for full bodies in motion. Specifically, our BEDLAM dataset contains monocular RGB videos with ground-truth 3D bodies in SMPL-X format. It includes a diversity of body shapes, motions, skin tones, hair, and clothing. The clothing is realistically simulated on the moving bodies using commercial clothing physics simulation. We render varying numbers of people in realistic scenes with varied lighting and camera motions. We then train various HPS regressors using BEDLAM and achieve state-of-the-art accuracy on real-image benchmarks despite training with synthetic data. We use BEDLAM to gain insights into what model design choices are important for accuracy. With good synthetic training data, we find that a basic method like HMR approaches the accuracy of the current SOTA method (CLIFF). BEDLAM is useful for a variety of tasks and all images, ground truth bodies, 3D clothing, support code, and more are available for research purposes. Additionally, we provide detailed information about our synthetic data generation pipeline, enabling others to generate their own datasets. See the project page: https://bedlam.is.tue.mpg.de/.