NeRF类比：基于示例的NeRFs视觉属性转移

NeRF Analogies: Example-Based Visual Attribute Transfer for NeRFs

February 13, 2024

作者: Michael Fischer, Zhengqin Li, Thu Nguyen-Phuoc, Aljaz Bozic, Zhao Dong, Carl Marshall, Tobias Ritschel

cs.AI

摘要

神经辐射场（NeRF）编码了场景的3D几何和外观之间的特定关系。我们在这里提出一个问题，即我们是否可以以一种语义上有意义的方式，将源NeRF的外观转移到目标3D几何上，使得生成的新NeRF保留目标几何但具有类似于源NeRF的外观。为此，我们将经典图像类比从2D图像推广到NeRF。我们利用来自大型预训练2D图像模型的语义特征驱动的语义亲和力进行对应转移，实现多视角一致的外观转移。我们的方法允许探索3D几何和外观的混搭产品空间。我们展示了我们的方法优于传统的基于风格化的方法，并且绝大多数用户更喜欢我们的方法而不是几种典型的基准方法。

English

A Neural Radiance Field (NeRF) encodes the specific relation of 3D geometry and appearance of a scene. We here ask the question whether we can transfer the appearance from a source NeRF onto a target 3D geometry in a semantically meaningful way, such that the resulting new NeRF retains the target geometry but has an appearance that is an analogy to the source NeRF. To this end, we generalize classic image analogies from 2D images to NeRFs. We leverage correspondence transfer along semantic affinity that is driven by semantic features from large, pre-trained 2D image models to achieve multi-view consistent appearance transfer. Our method allows exploring the mix-and-match product space of 3D geometry and appearance. We show that our method outperforms traditional stylization-based methods and that a large majority of users prefer our method over several typical baselines.