PointInfinity: 分辨率不变的点扩散模型

PointInfinity: Resolution-Invariant Point Diffusion Models

April 4, 2024

作者: Zixuan Huang, Justin Johnson, Shoubhik Debnath, James M. Rehg, Chao-Yuan Wu

cs.AI

摘要

我们提出了PointInfinity，这是一种高效的点云扩散模型系列。我们的核心思想是使用基于Transformer的架构，具有固定大小、分辨率不变的潜在表示。这使得能够在低分辨率点云上进行高效训练，同时允许在推断过程中生成高分辨率点云。更重要的是，我们展示了将测试时分辨率扩展到训练分辨率之上可以提高生成的点云和表面的保真度。我们分析了这一现象，并将其与扩散模型中常用的无分类器引导进行了联系，表明两者都允许在推断过程中权衡保真度和变异性。在CO3D上的实验表明，PointInfinity能够高效生成高分辨率点云（最多131k个点，比Point-E多31倍），并具有最先进的质量。

English

We present PointInfinity, an efficient family of point cloud diffusion models. Our core idea is to use a transformer-based architecture with a fixed-size, resolution-invariant latent representation. This enables efficient training with low-resolution point clouds, while allowing high-resolution point clouds to be generated during inference. More importantly, we show that scaling the test-time resolution beyond the training resolution improves the fidelity of generated point clouds and surfaces. We analyze this phenomenon and draw a link to classifier-free guidance commonly used in diffusion models, demonstrating that both allow trading off fidelity and variability during inference. Experiments on CO3D show that PointInfinity can efficiently generate high-resolution point clouds (up to 131k points, 31 times more than Point-E) with state-of-the-art quality.