神经安琪洛:高保真神经表面重建
Neuralangelo: High-Fidelity Neural Surface Reconstruction
June 5, 2023
作者: Zhaoshuo Li, Thomas Müller, Alex Evans, Russell H. Taylor, Mathias Unberath, Ming-Yu Liu, Chen-Hsuan Lin
cs.AI
摘要
神经表面重建已被证明在通过基于图像的神经渲染恢复密集3D表面方面非常强大。然而,当前方法在恢复真实场景的详细结构方面存在困难。为了解决这一问题,我们提出了Neuralangelo,它将多分辨率3D哈希网格的表示能力与神经表面渲染相结合。两个关键要素使我们的方法成为可能:(1) 用于计算高阶导数的数值梯度作为平滑操作,以及 (2) 在控制不同细节级别的哈希网格上进行由粗到细的优化。即使没有深度等辅助输入,Neuralangelo也能够从多视图图像中有效恢复密集的3D表面结构,其保真度显著超过先前的方法,从而实现了从RGB视频捕获中对大规模场景进行详细重建。
English
Neural surface reconstruction has been shown to be powerful for recovering
dense 3D surfaces via image-based neural rendering. However, current methods
struggle to recover detailed structures of real-world scenes. To address the
issue, we present Neuralangelo, which combines the representation power of
multi-resolution 3D hash grids with neural surface rendering. Two key
ingredients enable our approach: (1) numerical gradients for computing
higher-order derivatives as a smoothing operation and (2) coarse-to-fine
optimization on the hash grids controlling different levels of details. Even
without auxiliary inputs such as depth, Neuralangelo can effectively recover
dense 3D surface structures from multi-view images with fidelity significantly
surpassing previous methods, enabling detailed large-scale scene reconstruction
from RGB video captures.