GaussianCube：利用最优输运结构化高斯飞溅以用于3D生成建模。

GaussianCube: Structuring Gaussian Splatting using Optimal Transport for 3D Generative Modeling

March 28, 2024

作者: Bowen Zhang, Yiji Cheng, Jiaolong Yang, Chunyu Wang, Feng Zhao, Yansong Tang, Dong Chen, Baining Guo

cs.AI

摘要

3D高斯喷洒（GS）在3D拟合保真度和渲染速度方面取得了相当大的改进，超越了神经辐射场。然而，这种具有散布高斯函数的非结构化表示对生成建模构成了重大挑战。为解决这一问题，我们引入了GaussianCube，这是一种既强大又高效的结构化GS表示，适用于生成建模。我们首先提出了一种修改后的密度约束GS拟合算法，可以利用固定数量的自由高斯函数产生高质量的拟合结果，然后通过最优输运将高斯函数重新排列到预定义的体素网格中。结构化网格表示使我们能够在扩散生成建模中使用标准的3D U-Net作为骨干，而无需复杂的设计。在ShapeNet和OmniObject3D上进行的大量实验表明，我们的模型在定性和定量上均实现了最先进的生成结果，突显了GaussianCube作为强大且多功能的3D表示的潜力。

English

3D Gaussian Splatting (GS) have achieved considerable improvement over Neural Radiance Fields in terms of 3D fitting fidelity and rendering speed. However, this unstructured representation with scattered Gaussians poses a significant challenge for generative modeling. To address the problem, we introduce GaussianCube, a structured GS representation that is both powerful and efficient for generative modeling. We achieve this by first proposing a modified densification-constrained GS fitting algorithm which can yield high-quality fitting results using a fixed number of free Gaussians, and then re-arranging the Gaussians into a predefined voxel grid via Optimal Transport. The structured grid representation allows us to use standard 3D U-Net as our backbone in diffusion generative modeling without elaborate designs. Extensive experiments conducted on ShapeNet and OmniObject3D show that our model achieves state-of-the-art generation results both qualitatively and quantitatively, underscoring the potential of GaussianCube as a powerful and versatile 3D representation.