個性化您的 NeRF：透過局部-全域迭代訓練進行自適應源驅動的 3D 場景編輯

摘要

本文針對自適應來源驅動的3D場景編輯任務，提出了一個名為CustomNeRF的模型，該模型將文本描述或參考圖像統一作為編輯提示。然而，為了獲得符合編輯提示的期望編輯結果並不簡單，因為存在兩個重要挑戰，包括僅準確編輯前景區域以及在單視角參考圖像的情況下實現多視角一致性。為應對第一個挑戰，我們提出了一種名為Local-Global Iterative Editing（LGIE）的訓練方案，該方案在前景區域編輯和完整圖像編輯之間交替進行，旨在實現僅針對前景的操作，同時保留背景。對於第二個挑戰，我們還設計了一種基於類別的正則化方法，利用生成模型內的類別先驗來減輕基於圖像的編輯中不同視角之間的不一致問題。大量實驗表明，我們的CustomNeRF在各種真實場景下均能產生精確的編輯結果，無論是在文本驅動還是圖像驅動的情況下。

English

In this paper, we target the adaptive source driven 3D scene editing task by proposing a CustomNeRF model that unifies a text description or a reference image as the editing prompt. However, obtaining desired editing results conformed with the editing prompt is nontrivial since there exist two significant challenges, including accurate editing of only foreground regions and multi-view consistency given a single-view reference image. To tackle the first challenge, we propose a Local-Global Iterative Editing (LGIE) training scheme that alternates between foreground region editing and full-image editing, aimed at foreground-only manipulation while preserving the background. For the second challenge, we also design a class-guided regularization that exploits class priors within the generation model to alleviate the inconsistency problem among different views in image-driven editing. Extensive experiments show that our CustomNeRF produces precise editing results under various real scenes for both text- and image-driven settings.

個性化您的 NeRF：透過局部-全域迭代訓練進行自適應源驅動的 3D 場景編輯

Customize your NeRF: Adaptive Source Driven 3D Scene Editing via Local-Global Iterative Training

摘要

Support