ZeroShape：基於回歸的零樣本形狀重建

摘要

我們研究單張圖像零樣本3D形狀重建問題。最近的研究通過生成建模3D資產來學習零樣本形狀重建，但這些模型在訓練和推論時計算成本高昂。相比之下，這個問題的傳統方法是基於回歸的，其中訓練確定性模型直接回歸物體形狀。這種回歸方法比生成方法具有更高的計算效率。這帶出一個自然問題：生成建模對於高性能是必要的嗎，或者相反，基於回歸的方法仍然具有競爭力？為了回答這個問題，我們設計了一個強大的基於回歸的模型，稱為ZeroShape，基於這一領域的收斂發現和一個新的洞察。我們還精心挑選了一個大型的現實世界評估基準，其中包含來自三個不同現實世界3D資料集的物體。這個評估基準更加多樣化，比先前的作品用於定量評估模型的數據量大一個數量級，旨在減少我們領域中的評估變異性。我們展示ZeroShape不僅實現了優越的性能，還顯著展示了更高的計算和數據效率。

English

We study the problem of single-image zero-shot 3D shape reconstruction. Recent works learn zero-shot shape reconstruction through generative modeling of 3D assets, but these models are computationally expensive at train and inference time. In contrast, the traditional approach to this problem is regression-based, where deterministic models are trained to directly regress the object shape. Such regression methods possess much higher computational efficiency than generative methods. This raises a natural question: is generative modeling necessary for high performance, or conversely, are regression-based approaches still competitive? To answer this, we design a strong regression-based model, called ZeroShape, based on the converging findings in this field and a novel insight. We also curate a large real-world evaluation benchmark, with objects from three different real-world 3D datasets. This evaluation benchmark is more diverse and an order of magnitude larger than what prior works use to quantitatively evaluate their models, aiming at reducing the evaluation variance in our field. We show that ZeroShape not only achieves superior performance over state-of-the-art methods, but also demonstrates significantly higher computational and data efficiency.

ZeroShape：基於回歸的零樣本形狀重建

ZeroShape: Regression-based Zero-shot Shape Reconstruction

摘要

Support