使用高斯點陣將文本轉換為3D

摘要

本文介紹了基於高斯點陣的文本轉3D生成（GSGEN）方法，這是一種用於生成高質量3D物體的新方法。先前的方法由於缺乏3D先驗和適當表示，導致幾何不準確且保真度有限。我們利用3D高斯點陣，這是一種最新的先進表示方法，通過利用明確的特性來克服現有缺點，實現對3D先驗的整合。具體而言，我們的方法採用漸進優化策略，包括幾何優化階段和外觀細化階段。在幾何優化中，建立了一個粗略表示，根據3D幾何先驗以及普通的2D SDS損失，確保一個合理且符合3D的粗略形狀。隨後，所獲得的高斯點陣經過迭代細化以豐富細節。在這個階段，我們通過基於緊湊性的致密化增加高斯點陣的數量，以增強連續性並提高保真度。通過這些設計，我們的方法可以生成帶有精細細節和更準確幾何的3D內容。廣泛的評估證明了我們的方法的有效性，特別是對於捕捉高頻組件。視頻結果可在https://gsgen3d.github.io 上查看。我們的代碼可在https://github.com/gsgen3d/gsgen 上找到。

English

In this paper, we present Gaussian Splatting based text-to-3D generation (GSGEN), a novel approach for generating high-quality 3D objects. Previous methods suffer from inaccurate geometry and limited fidelity due to the absence of 3D prior and proper representation. We leverage 3D Gaussian Splatting, a recent state-of-the-art representation, to address existing shortcomings by exploiting the explicit nature that enables the incorporation of 3D prior. Specifically, our method adopts a progressive optimization strategy, which includes a geometry optimization stage and an appearance refinement stage. In geometry optimization, a coarse representation is established under a 3D geometry prior along with the ordinary 2D SDS loss, ensuring a sensible and 3D-consistent rough shape. Subsequently, the obtained Gaussians undergo an iterative refinement to enrich details. In this stage, we increase the number of Gaussians by compactness-based densification to enhance continuity and improve fidelity. With these designs, our approach can generate 3D content with delicate details and more accurate geometry. Extensive evaluations demonstrate the effectiveness of our method, especially for capturing high-frequency components. Video results are provided at https://gsgen3d.github.io. Our code is available at https://github.com/gsgen3d/gsgen

使用高斯點陣將文本轉換為3D

Text-to-3D using Gaussian Splatting

摘要

Support