DreamCatalyst：透過控制可編輯性和身份保護，快速高質量的3D編輯。

DreamCatalyst: Fast and High-Quality 3D Editing via Controlling Editability and Identity Preservation

July 16, 2024

作者: Jiwook Kim, Seonho Lee, Jaeyo Shin, Jiho Choi, Hyunjung Shim

cs.AI

摘要

由於其固有的3D一致性，得分蒸餾採樣（SDS）已成為文本驅動的3D編輯任務中的一個有效框架。然而，現有基於SDS的3D編輯方法存在著長時間的訓練和低質量結果的問題，主要是因為這些方法偏離了擴散模型的採樣動態。在本文中，我們提出了DreamCatalyst，一個新穎的框架，將基於SDS的編輯解釋為擴散反向過程。我們的目標函數考慮了採樣動態，從而使DreamCatalyst的優化過程成為編輯任務中擴散反向過程的近似。DreamCatalyst的目標是減少訓練時間並提高編輯質量。DreamCatalyst提供了兩種模式：（1）更快速的模式，僅需約25分鐘編輯NeRF場景，（2）高質量模式，在不到70分鐘內產生優越結果。具體而言，我們的高質量模式在速度和質量方面均優於當前NeRF編輯方法的最新技術。更多詳細結果請參見我們的項目頁面：https://dream-catalyst.github.io。

English

Score distillation sampling (SDS) has emerged as an effective framework in text-driven 3D editing tasks due to its inherent 3D consistency. However, existing SDS-based 3D editing methods suffer from extensive training time and lead to low-quality results, primarily because these methods deviate from the sampling dynamics of diffusion models. In this paper, we propose DreamCatalyst, a novel framework that interprets SDS-based editing as a diffusion reverse process. Our objective function considers the sampling dynamics, thereby making the optimization process of DreamCatalyst an approximation of the diffusion reverse process in editing tasks. DreamCatalyst aims to reduce training time and improve editing quality. DreamCatalyst presents two modes: (1) a faster mode, which edits the NeRF scene in only about 25 minutes, and (2) a high-quality mode, which produces superior results in less than 70 minutes. Specifically, our high-quality mode outperforms current state-of-the-art NeRF editing methods both in terms of speed and quality. See more extensive results on our project page: https://dream-catalyst.github.io.