MeshLRM：用于高质量网格的大型重建模型

MeshLRM: Large Reconstruction Model for High-Quality Mesh

April 18, 2024

作者: Xinyue Wei, Kai Zhang, Sai Bi, Hao Tan, Fujun Luan, Valentin Deschaintre, Kalyan Sunkavalli, Hao Su, Zexiang Xu

cs.AI

摘要

我们提出了MeshLRM，这是一种基于LRM的新方法，可以在不到一秒的时间内仅从四幅输入图像中重建出高质量的网格。与先前专注于基于NeRF的重建的大型重建模型（LRMs）不同，MeshLRM在LRM框架内结合了可微的网格提取和渲染。这使得通过微调预训练的NeRF LRM与网格渲染实现端到端的网格重建成为可能。此外，我们通过简化先前LRMs中的若干复杂设计来改进LRM架构。MeshLRM的NeRF初始化经过低分辨率和高分辨率图像的顺序训练；这种新的LRM训练策略实现了显著更快的收敛速度，从而以更少的计算量实现更好的质量。我们的方法实现了从稀疏视图输入中的最先进的网格重建，同时还支持许多下游应用，包括文本到3D和单图像到3D生成。项目页面：https://sarahweiii.github.io/meshlrm/

English

We propose MeshLRM, a novel LRM-based approach that can reconstruct a high-quality mesh from merely four input images in less than one second. Different from previous large reconstruction models (LRMs) that focus on NeRF-based reconstruction, MeshLRM incorporates differentiable mesh extraction and rendering within the LRM framework. This allows for end-to-end mesh reconstruction by fine-tuning a pre-trained NeRF LRM with mesh rendering. Moreover, we improve the LRM architecture by simplifying several complex designs in previous LRMs. MeshLRM's NeRF initialization is sequentially trained with low- and high-resolution images; this new LRM training strategy enables significantly faster convergence and thereby leads to better quality with less compute. Our approach achieves state-of-the-art mesh reconstruction from sparse-view inputs and also allows for many downstream applications, including text-to-3D and single-image-to-3D generation. Project page: https://sarahweiii.github.io/meshlrm/