Freditor：透過頻率分解的高保真度和可轉移性 NeRF 編輯

Freditor: High-Fidelity and Transferable NeRF Editing by Frequency Decomposition

April 3, 2024

作者: Yisheng He, Weihao Yuan, Siyu Zhu, Zilong Dong, Liefeng Bo, Qixing Huang

cs.AI

摘要

本文通過頻率分解實現高保真、可轉移的 NeRF 編輯。最近的 NeRF 編輯流程將 2D 風格化結果提升至 3D 場景，但存在模糊結果問題，並且無法捕捉由於 2D 編輯不一致而導致的細節結構。我們的關鍵洞察是圖像的低頻分量在編輯後與高頻部分相比更具多視角一致性。此外，外觀風格主要展現在低頻分量上，而內容細節尤其存在於高頻部分。這促使我們在低頻分量上進行編輯，從而產生高保真的編輯場景。此外，編輯是在低頻特徵空間中進行的，實現穩定的強度控制和新穎的場景轉移。在逼真數據集上進行的全面實驗證明了高保真和可轉移的 NeRF 編輯的卓越性能。項目頁面位於 https://aigc3d.github.io/freditor。

English

This paper enables high-fidelity, transferable NeRF editing by frequency decomposition. Recent NeRF editing pipelines lift 2D stylization results to 3D scenes while suffering from blurry results, and fail to capture detailed structures caused by the inconsistency between 2D editings. Our critical insight is that low-frequency components of images are more multiview-consistent after editing compared with their high-frequency parts. Moreover, the appearance style is mainly exhibited on the low-frequency components, and the content details especially reside in high-frequency parts. This motivates us to perform editing on low-frequency components, which results in high-fidelity edited scenes. In addition, the editing is performed in the low-frequency feature space, enabling stable intensity control and novel scene transfer. Comprehensive experiments conducted on photorealistic datasets demonstrate the superior performance of high-fidelity and transferable NeRF editing. The project page is at https://aigc3d.github.io/freditor.