DreamPolisher：通過幾何擴散朝向高質量的文本到3D生成

DreamPolisher: Towards High-Quality Text-to-3D Generation via Geometric Diffusion

March 25, 2024

作者: Yuanze Lin, Ronald Clark, Philip Torr

cs.AI

摘要

我們提出了DreamPolisher，一種新穎的基於高斯擴散的方法，具有幾何引導，旨在從文本描述中學習跨視圖一致性和細緻細節。儘管最近在文本到3D生成方法上取得了令人鼓舞的進展，但主流方法通常無法確保視圖一致性和紋理豐富性。這個問題尤其突出於僅使用文本輸入的方法。為了解決這個問題，我們提出了一種基於兩階段高斯擴散的方法，以實現視圖之間的幾何一致性。首先，粗略的3D生成經過幾何優化進行細化。隨後，我們使用一個ControlNet驅動的細化器，結合幾何一致性術語，來提高生成的3D資產的紋理保真度和整體一致性。通過涵蓋各種物體類別的多樣文本提示的實證評估顯示了DreamPolisher在生成一致且逼真的3D物體方面的有效性，與文本指示的語義密切相符。

English

We present DreamPolisher, a novel Gaussian Splatting based method with geometric guidance, tailored to learn cross-view consistency and intricate detail from textual descriptions. While recent progress on text-to-3D generation methods have been promising, prevailing methods often fail to ensure view-consistency and textural richness. This problem becomes particularly noticeable for methods that work with text input alone. To address this, we propose a two-stage Gaussian Splatting based approach that enforces geometric consistency among views. Initially, a coarse 3D generation undergoes refinement via geometric optimization. Subsequently, we use a ControlNet driven refiner coupled with the geometric consistency term to improve both texture fidelity and overall consistency of the generated 3D asset. Empirical evaluations across diverse textual prompts spanning various object categories demonstrate the efficacy of DreamPolisher in generating consistent and realistic 3D objects, aligning closely with the semantics of the textual instructions.