ICON：联合姿态和辐射场的增量置信度优化

ICON: Incremental CONfidence for Joint Pose and Radiance Field Optimization

January 17, 2024

作者: Weiyao Wang, Pierre Gleize, Hao Tang, Xingyu Chen, Kevin J Liang, Matt Feiszli

cs.AI

摘要

神经辐射场（Neural Radiance Fields，NeRF）在给定一组2D图像时表现出卓越的新视角合成（Novel View Synthesis，NVS）性能。然而，NeRF的训练需要准确的摄像机姿势，通常通过运动结构（Structure-from-Motion，SfM）流程获得。最近的研究尝试放宽这一约束，但它们仍然经常依赖可以优化的良好初始姿势。在这里，我们旨在消除对姿势初始化的要求。我们提出了增量置信（Incremental CONfidence，ICON），这是一种用于从2D视频帧训练NeRF的优化过程。ICON仅假设平滑的摄像机运动来估计姿势的初始猜测。此外，ICON引入了“置信度”：这是一种用于动态重新加权梯度的自适应模型质量度量。ICON依赖于高置信度姿势来学习NeRF，并依赖于高置信度的3D结构（由NeRF编码）来学习姿势。我们展示了，ICON在没有先前姿势初始化的情况下，在CO3D和HO3D方面均比使用SfM姿势的方法表现出更优异的性能。

English

Neural Radiance Fields (NeRF) exhibit remarkable performance for Novel View Synthesis (NVS) given a set of 2D images. However, NeRF training requires accurate camera pose for each input view, typically obtained by Structure-from-Motion (SfM) pipelines. Recent works have attempted to relax this constraint, but they still often rely on decent initial poses which they can refine. Here we aim at removing the requirement for pose initialization. We present Incremental CONfidence (ICON), an optimization procedure for training NeRFs from 2D video frames. ICON only assumes smooth camera motion to estimate initial guess for poses. Further, ICON introduces ``confidence": an adaptive measure of model quality used to dynamically reweight gradients. ICON relies on high-confidence poses to learn NeRF, and high-confidence 3D structure (as encoded by NeRF) to learn poses. We show that ICON, without prior pose initialization, achieves superior performance in both CO3D and HO3D versus methods which use SfM pose.