梦想打磨者：通过几何扩散实现高质量文本到3D生成

DreamPolisher: Towards High-Quality Text-to-3D Generation via Geometric Diffusion

March 25, 2024

作者: Yuanze Lin, Ronald Clark, Philip Torr

cs.AI

摘要

我们提出了DreamPolisher，这是一种基于高斯光滑的方法，具有几何引导，旨在从文本描述中学习跨视图一致性和复杂细节。虽然最近关于文本到3D生成方法的进展令人鼓舞，但主流方法通常无法确保视图一致性和纹理丰富性。对于仅使用文本输入的方法，这个问题尤为明显。为解决这一问题，我们提出了一种基于两阶段高斯光滑的方法，强调视图之间的几何一致性。首先，粗略的3D生成经过几何优化进行细化。随后，我们使用一个由ControlNet驱动的细化器，结合几何一致性项，来提高生成的3D资产的纹理保真度和整体一致性。通过跨越各种物体类别的多样文本提示进行的实证评估表明，DreamPolisher在生成一致且逼真的3D物体方面表现出显著效果，与文本指令的语义紧密契合。

English

We present DreamPolisher, a novel Gaussian Splatting based method with geometric guidance, tailored to learn cross-view consistency and intricate detail from textual descriptions. While recent progress on text-to-3D generation methods have been promising, prevailing methods often fail to ensure view-consistency and textural richness. This problem becomes particularly noticeable for methods that work with text input alone. To address this, we propose a two-stage Gaussian Splatting based approach that enforces geometric consistency among views. Initially, a coarse 3D generation undergoes refinement via geometric optimization. Subsequently, we use a ControlNet driven refiner coupled with the geometric consistency term to improve both texture fidelity and overall consistency of the generated 3D asset. Empirical evaluations across diverse textual prompts spanning various object categories demonstrate the efficacy of DreamPolisher in generating consistent and realistic 3D objects, aligning closely with the semantics of the textual instructions.