ChatPaper.aiChatPaper

DreamCatalyst:通过控制可编辑性和身份保留实现快速高质量的3D编辑。

DreamCatalyst: Fast and High-Quality 3D Editing via Controlling Editability and Identity Preservation

July 16, 2024
作者: Jiwook Kim, Seonho Lee, Jaeyo Shin, Jiho Choi, Hyunjung Shim
cs.AI

摘要

由于其固有的三维一致性,得分蒸馏采样(SDS)已成为文本驱动的三维编辑任务中的有效框架。然而,现有基于SDS的三维编辑方法存在训练时间长且导致低质量结果的问题,主要是因为这些方法偏离了扩散模型的采样动态。在本文中,我们提出了DreamCatalyst,这是一个新颖的框架,将基于SDS的编辑解释为扩散反向过程。我们的目标函数考虑了采样动态,从而使DreamCatalyst的优化过程成为编辑任务中扩散反向过程的近似。DreamCatalyst的目标是减少训练时间并提高编辑质量。DreamCatalyst提供两种模式:(1)更快速的模式,仅需约25分钟编辑NeRF场景;(2)高质量模式,在不到70分钟内产生优越结果。具体而言,我们的高质量模式在速度和质量方面均优于当前NeRF编辑方法的最新技术。更多详尽结果请查看我们的项目页面:https://dream-catalyst.github.io。
English
Score distillation sampling (SDS) has emerged as an effective framework in text-driven 3D editing tasks due to its inherent 3D consistency. However, existing SDS-based 3D editing methods suffer from extensive training time and lead to low-quality results, primarily because these methods deviate from the sampling dynamics of diffusion models. In this paper, we propose DreamCatalyst, a novel framework that interprets SDS-based editing as a diffusion reverse process. Our objective function considers the sampling dynamics, thereby making the optimization process of DreamCatalyst an approximation of the diffusion reverse process in editing tasks. DreamCatalyst aims to reduce training time and improve editing quality. DreamCatalyst presents two modes: (1) a faster mode, which edits the NeRF scene in only about 25 minutes, and (2) a high-quality mode, which produces superior results in less than 70 minutes. Specifically, our high-quality mode outperforms current state-of-the-art NeRF editing methods both in terms of speed and quality. See more extensive results on our project page: https://dream-catalyst.github.io.
PDF122November 28, 2024