AGG：摊销生成式3D高斯模型用于从单张图像到3D

摘要

随着对自动三维内容创建流程日益增长的需求，研究了各种三维表示形式，以从单个图像生成三维对象。由于其出色的渲染效率，基于三维高斯飞溅的模型最近在三维重建和生成方面表现出色。基于三维高斯飞溅的图像到三维生成方法通常是基于优化的，需要许多计算昂贵的分数蒸馏步骤。为了克服这些挑战，我们引入了一种摊销生成式三维高斯框架（AGG），可以即时从单个图像生成三维高斯，消除了每个实例优化的需要。利用中间混合表示，AGG将三维高斯位置的生成和其他外观属性的联合优化进行了分解。此外，我们提出了一个级联流程，首先生成三维数据的粗略表示，然后利用三维高斯超分辨率模块对其进行上采样。我们的方法与现有基于优化的三维高斯框架和利用其他三维表示的基于采样的流程进行了评估，AGG在生成能力上在定性和定量上展示出竞争力，同时速度快几个数量级。项目页面：https://ir1d.github.io/AGG/

English

Given the growing need for automatic 3D content creation pipelines, various 3D representations have been studied to generate 3D objects from a single image. Due to its superior rendering efficiency, 3D Gaussian splatting-based models have recently excelled in both 3D reconstruction and generation. 3D Gaussian splatting approaches for image to 3D generation are often optimization-based, requiring many computationally expensive score-distillation steps. To overcome these challenges, we introduce an Amortized Generative 3D Gaussian framework (AGG) that instantly produces 3D Gaussians from a single image, eliminating the need for per-instance optimization. Utilizing an intermediate hybrid representation, AGG decomposes the generation of 3D Gaussian locations and other appearance attributes for joint optimization. Moreover, we propose a cascaded pipeline that first generates a coarse representation of the 3D data and later upsamples it with a 3D Gaussian super-resolution module. Our method is evaluated against existing optimization-based 3D Gaussian frameworks and sampling-based pipelines utilizing other 3D representations, where AGG showcases competitive generation abilities both qualitatively and quantitatively while being several orders of magnitude faster. Project page: https://ir1d.github.io/AGG/

AGG：摊销生成式3D高斯模型用于从单张图像到3D

AGG: Amortized Generative 3D Gaussians for Single Image to 3D

摘要

Support