GarVerseLOD：使用具有不同細節層次的數據集，從單一野外圖像實現高保真度的3D服裝重建

GarVerseLOD: High-Fidelity 3D Garment Reconstruction from a Single In-the-Wild Image using a Dataset with Levels of Details

November 5, 2024

作者: Zhongjin Luo, Haolin Liu, Chenghong Li, Wanghao Du, Zirong Jin, Wanhu Sun, Yinyu Nie, Weikai Chen, Xiaoguang Han

cs.AI

摘要

神經隱式函數為從多張甚至單張圖像中的服裝人體數位化的最新技術帶來了令人印象深刻的進展。然而，儘管取得了進步，目前的方法仍然難以泛化到具有複雜布料變形和身體姿勢的未見過圖像。在這項工作中，我們提出了GarVerseLOD，這是一個新的數據集和框架，為從單張無限制圖像實現高保真度的3D服裝重建打開了道路。受大型生成模型最近成功的啟發，我們認為應對泛化挑戰的關鍵之一在於3D服裝數據的數量和質量。為此，GarVerseLOD收集了由專業藝術家手動創建的具有精細幾何細節的6,000個高質量布料模型。除了訓練數據的規模外，我們觀察到幾何的解耦細節可能在提升模型的泛化能力和推理準確性方面發揮重要作用。因此，我們將GarVerseLOD打造為一個具有細節層次（LOD）的分層數據集，從無細節的風格化形狀到與像素對齊細節的姿勢混合服裝。這使我們能夠通過將推理分解為更容易的任務，每個任務的搜索空間縮小，使這個高度不受約束的問題變得可控。為了確保GarVerseLOD能夠很好地泛化到野外圖像，我們提出了一種基於條件擴散模型的新標記範式，為每個服裝模型生成大量具有高照片逼真度的配對圖像。我們在大量野外圖像上評估了我們的方法。實驗結果表明，GarVerseLOD可以生成獨立的服裝部件，其質量顯著優於先前的方法。項目頁面：https://garverselod.github.io/

English

Neural implicit functions have brought impressive advances to the state-of-the-art of clothed human digitization from multiple or even single images. However, despite the progress, current arts still have difficulty generalizing to unseen images with complex cloth deformation and body poses. In this work, we present GarVerseLOD, a new dataset and framework that paves the way to achieving unprecedented robustness in high-fidelity 3D garment reconstruction from a single unconstrained image. Inspired by the recent success of large generative models, we believe that one key to addressing the generalization challenge lies in the quantity and quality of 3D garment data. Towards this end, GarVerseLOD collects 6,000 high-quality cloth models with fine-grained geometry details manually created by professional artists. In addition to the scale of training data, we observe that having disentangled granularities of geometry can play an important role in boosting the generalization capability and inference accuracy of the learned model. We hence craft GarVerseLOD as a hierarchical dataset with levels of details (LOD), spanning from detail-free stylized shape to pose-blended garment with pixel-aligned details. This allows us to make this highly under-constrained problem tractable by factorizing the inference into easier tasks, each narrowed down with smaller searching space. To ensure GarVerseLOD can generalize well to in-the-wild images, we propose a novel labeling paradigm based on conditional diffusion models to generate extensive paired images for each garment model with high photorealism. We evaluate our method on a massive amount of in-the-wild images. Experimental results demonstrate that GarVerseLOD can generate standalone garment pieces with significantly better quality than prior approaches. Project page: https://garverselod.github.io/