BEDLAM：展示細緻逼真動態的身體合成數據集

BEDLAM: A Synthetic Dataset of Bodies Exhibiting Detailed Lifelike Animated Motion

June 29, 2023

作者: Michael J. Black, Priyanka Patel, Joachim Tesch, Jinlong Yang

cs.AI

摘要

我們首次展示，僅使用合成數據訓練的神經網絡在解決從真實圖像中估計3D人體姿態和形狀（HPS）問題時實現了最先進的準確性。先前的合成數據集規模較小、不現實，或缺乏真實的服裝。實現足夠的真實感並非易事，我們展示了如何在運動中實現全身的真實感。具體來說，我們的BEDLAM數據集包含以SMPL-X格式呈現的單眼RGB視頻中的真實3D人體。它包含多種身體形狀、動作、膚色、頭髮和服裝。使用商業服裝物理模擬在移動的人體上逼真模擬服裝。我們在逼真場景中以不同的光線和相機運動渲染不同數量的人。然後，我們使用BEDLAM訓練各種HPS回歸器，在真實圖像基準上實現了最先進的準確性，儘管是使用合成數據進行訓練。我們使用BEDLAM來深入了解哪些模型設計選擇對準確性至關重要。通過良好的合成訓練數據，我們發現像HMR這樣的基本方法可以接近當前的最先進方法（CLIFF）的準確性。BEDLAM對各種任務都很有用，所有圖像、真實人體、3D服裝、支持代碼等均可供研究目的使用。此外，我們提供有關我們合成數據生成流程的詳細信息，使其他人能夠生成自己的數據集。請查看項目頁面：https://bedlam.is.tue.mpg.de/。

English

We show, for the first time, that neural networks trained only on synthetic data achieve state-of-the-art accuracy on the problem of 3D human pose and shape (HPS) estimation from real images. Previous synthetic datasets have been small, unrealistic, or lacked realistic clothing. Achieving sufficient realism is non-trivial and we show how to do this for full bodies in motion. Specifically, our BEDLAM dataset contains monocular RGB videos with ground-truth 3D bodies in SMPL-X format. It includes a diversity of body shapes, motions, skin tones, hair, and clothing. The clothing is realistically simulated on the moving bodies using commercial clothing physics simulation. We render varying numbers of people in realistic scenes with varied lighting and camera motions. We then train various HPS regressors using BEDLAM and achieve state-of-the-art accuracy on real-image benchmarks despite training with synthetic data. We use BEDLAM to gain insights into what model design choices are important for accuracy. With good synthetic training data, we find that a basic method like HMR approaches the accuracy of the current SOTA method (CLIFF). BEDLAM is useful for a variety of tasks and all images, ground truth bodies, 3D clothing, support code, and more are available for research purposes. Additionally, we provide detailed information about our synthetic data generation pipeline, enabling others to generate their own datasets. See the project page: https://bedlam.is.tue.mpg.de/.