GaussianCube：使用最佳運輸結構化高斯飛濺，用於3D生成建模

GaussianCube: Structuring Gaussian Splatting using Optimal Transport for 3D Generative Modeling

March 28, 2024

作者: Bowen Zhang, Yiji Cheng, Jiaolong Yang, Chunyu Wang, Feng Zhao, Yansong Tang, Dong Chen, Baining Guo

cs.AI

摘要

3D 高斯飛濺（GS）在 3D 擬合精度和渲染速度方面已經比神經輻射場取得了顯著的改進。然而，這種具有分散高斯的非結構化表示對於生成建模構成了重大挑戰。為了解決這個問題，我們引入了 GaussianCube，這是一種結構化的 GS 表示，對於生成建模既強大又高效。我們首先提出了一種修改的密度約束 GS 擬合算法，可以使用固定數量的自由高斯獲得高質量的擬合結果，然後通過最優運輸將高斯重新排列到預定義的體素網格中。結構化網格表示使我們能夠在擴散生成建模中使用標準的 3D U-Net 作為骨幹，而無需精心設計。在 ShapeNet 和 OmniObject3D 上進行的大量實驗表明，我們的模型在質量和量化方面均取得了最先進的生成結果，凸顯了 GaussianCube 作為一種強大且多功能的 3D 表示的潛力。

English

3D Gaussian Splatting (GS) have achieved considerable improvement over Neural Radiance Fields in terms of 3D fitting fidelity and rendering speed. However, this unstructured representation with scattered Gaussians poses a significant challenge for generative modeling. To address the problem, we introduce GaussianCube, a structured GS representation that is both powerful and efficient for generative modeling. We achieve this by first proposing a modified densification-constrained GS fitting algorithm which can yield high-quality fitting results using a fixed number of free Gaussians, and then re-arranging the Gaussians into a predefined voxel grid via Optimal Transport. The structured grid representation allows us to use standard 3D U-Net as our backbone in diffusion generative modeling without elaborate designs. Extensive experiments conducted on ShapeNet and OmniObject3D show that our model achieves state-of-the-art generation results both qualitatively and quantitatively, underscoring the potential of GaussianCube as a powerful and versatile 3D representation.