UniPre3D：基於跨模態高斯濺射的三維點雲模型統一預訓練

摘要

点云数据的尺度多样性为开发统一的三维视觉表示学习技术带来了显著挑战。目前，统一的3D模型较少，且现有的预训练方法无法对物体级和场景级点云均等有效。本文中，我们提出了UniPre3D，这是首个能够无缝应用于任何尺度点云及任何架构3D模型的统一预训练方法。我们的方法通过预测高斯基元作为预训练任务，并采用可微分高斯溅射进行图像渲染，实现了精确的像素级监督和端到端优化。为了进一步调控预训练任务的复杂性并引导模型关注几何结构，我们整合了预训练图像模型中的2D特征，以融入成熟的纹理知识。我们通过使用多种点云模型作为骨干，在广泛的物体级和场景级任务上进行了大量实验，验证了所提出方法的普适有效性。代码可在https://github.com/wangzy22/UniPre3D获取。

English

The scale diversity of point cloud data presents significant challenges in developing unified representation learning techniques for 3D vision. Currently, there are few unified 3D models, and no existing pre-training method is equally effective for both object- and scene-level point clouds. In this paper, we introduce UniPre3D, the first unified pre-training method that can be seamlessly applied to point clouds of any scale and 3D models of any architecture. Our approach predicts Gaussian primitives as the pre-training task and employs differentiable Gaussian splatting to render images, enabling precise pixel-level supervision and end-to-end optimization. To further regulate the complexity of the pre-training task and direct the model's focus toward geometric structures, we integrate 2D features from pre-trained image models to incorporate well-established texture knowledge. We validate the universal effectiveness of our proposed method through extensive experiments across a variety of object- and scene-level tasks, using diverse point cloud models as backbones. Code is available at https://github.com/wangzy22/UniPre3D.

UniPre3D：基於跨模態高斯濺射的三維點雲模型統一預訓練

UniPre3D: Unified Pre-training of 3D Point Cloud Models with Cross-Modal Gaussian Splatting

摘要

Support