TIP-Editor：一個準確的3D編輯器，可以根據文本提示和圖像提示進行操作。

摘要

由於其便利性和用戶友好性，基於文本驅動的3D場景編輯引起了廣泛關注。然而，由於文本描述的固有限制，現有方法仍然缺乏對指定外觀和位置進行準確控制的能力。為此，我們提出了一個3D場景編輯框架TIPEditor，該框架接受文本和圖像提示以及3D邊界框來指定編輯區域。通過圖像提示，用戶可以方便地指定目標內容的詳細外觀/風格，以補充文本描述，從而實現對外觀的準確控制。具體來說，TIP-Editor採用逐步的2D個性化策略，以更好地學習現有場景和參考圖像的表示，其中提出了一種定位損失，以鼓勵按照邊界框指定的方式正確放置物體。此外，TIPEditor利用明確靈活的3D高斯擴散作為3D表示，以促進局部編輯，同時保持背景不變。大量實驗表明，TIP-Editor能夠在指定邊界框區域內根據文本和圖像提示進行準確編輯，始終優於基準線在編輯質量和對提示的對齊方面，在質量和量化方面都有優異表現。

English

Text-driven 3D scene editing has gained significant attention owing to its convenience and user-friendliness. However, existing methods still lack accurate control of the specified appearance and location of the editing result due to the inherent limitations of the text description. To this end, we propose a 3D scene editing framework, TIPEditor, that accepts both text and image prompts and a 3D bounding box to specify the editing region. With the image prompt, users can conveniently specify the detailed appearance/style of the target content in complement to the text description, enabling accurate control of the appearance. Specifically, TIP-Editor employs a stepwise 2D personalization strategy to better learn the representation of the existing scene and the reference image, in which a localization loss is proposed to encourage correct object placement as specified by the bounding box. Additionally, TIPEditor utilizes explicit and flexible 3D Gaussian splatting as the 3D representation to facilitate local editing while keeping the background unchanged. Extensive experiments have demonstrated that TIP-Editor conducts accurate editing following the text and image prompts in the specified bounding box region, consistently outperforming the baselines in editing quality, and the alignment to the prompts, qualitatively and quantitatively.

TIP-Editor：一個準確的3D編輯器，可以根據文本提示和圖像提示進行操作。

TIP-Editor: An Accurate 3D Editor Following Both Text-Prompts And Image-Prompts

摘要

Support