MeshLRM：用於高質量網格的大型重建模型

MeshLRM: Large Reconstruction Model for High-Quality Mesh

April 18, 2024

作者: Xinyue Wei, Kai Zhang, Sai Bi, Hao Tan, Fujun Luan, Valentin Deschaintre, Kalyan Sunkavalli, Hao Su, Zexiang Xu

cs.AI

摘要

我們提出了MeshLRM，一種基於LRM的新方法，可以在不到一秒的時間內僅從四張輸入圖像重建出高質量網格。與先前專注於NeRF-based重建的大型重建模型（LRMs）不同，MeshLRM將可微網格提取和渲染融入LRM框架中。這使得可以通過微調預先訓練的NeRF LRM並進行網格渲染來進行端到端的網格重建。此外，我們通過簡化先前LRMs中的幾個複雜設計來改進LRM架構。MeshLRM的NeRF初始化是通過低分辨率和高分辨率圖像進行順序訓練的；這種新的LRM訓練策略實現了顯著更快的收斂，從而在更少的計算量下獲得更好的質量。我們的方法從稀疏視圖輸入中實現了最先進的網格重建，同時還支持許多下游應用，包括文本到3D和單圖像到3D生成。項目頁面：https://sarahweiii.github.io/meshlrm/

English

We propose MeshLRM, a novel LRM-based approach that can reconstruct a high-quality mesh from merely four input images in less than one second. Different from previous large reconstruction models (LRMs) that focus on NeRF-based reconstruction, MeshLRM incorporates differentiable mesh extraction and rendering within the LRM framework. This allows for end-to-end mesh reconstruction by fine-tuning a pre-trained NeRF LRM with mesh rendering. Moreover, we improve the LRM architecture by simplifying several complex designs in previous LRMs. MeshLRM's NeRF initialization is sequentially trained with low- and high-resolution images; this new LRM training strategy enables significantly faster convergence and thereby leads to better quality with less compute. Our approach achieves state-of-the-art mesh reconstruction from sparse-view inputs and also allows for many downstream applications, including text-to-3D and single-image-to-3D generation. Project page: https://sarahweiii.github.io/meshlrm/