NeRF 比擬：基於範例的視覺屬性轉移方法，適用於 NeRF。

NeRF Analogies: Example-Based Visual Attribute Transfer for NeRFs

February 13, 2024

作者: Michael Fischer, Zhengqin Li, Thu Nguyen-Phuoc, Aljaz Bozic, Zhao Dong, Carl Marshall, Tobias Ritschel

cs.AI

摘要

神經輻射場（Neural Radiance Field，NeRF）編碼了場景的3D幾何和外觀之間的特定關係。我們在這裡探討一個問題，即我們是否可以以一種語義上有意義的方式，將源NeRF的外觀轉移到目標3D幾何上，使得結果的新NeRF保留目標幾何，但具有與源NeRF類似的外觀。為此，我們將經典的圖像類比從2D圖像推廣到NeRF。我們利用來自大型預訓練2D圖像模型的語義特徵驅動的語義相似性轉移，實現多視角一致的外觀轉移。我們的方法允許探索3D幾何和外觀的混合匹配產品空間。我們展示了我們的方法優於傳統的基於風格化的方法，並且絕大多數用戶更喜歡我們的方法而不是幾種典型的基準方法。

English

A Neural Radiance Field (NeRF) encodes the specific relation of 3D geometry and appearance of a scene. We here ask the question whether we can transfer the appearance from a source NeRF onto a target 3D geometry in a semantically meaningful way, such that the resulting new NeRF retains the target geometry but has an appearance that is an analogy to the source NeRF. To this end, we generalize classic image analogies from 2D images to NeRFs. We leverage correspondence transfer along semantic affinity that is driven by semantic features from large, pre-trained 2D image models to achieve multi-view consistent appearance transfer. Our method allows exploring the mix-and-match product space of 3D geometry and appearance. We show that our method outperforms traditional stylization-based methods and that a large majority of users prefer our method over several typical baselines.