UniPre3D：基于跨模态高斯溅射的3D点云模型统一预训练

摘要

点云数据的尺度多样性为开发统一的3D视觉表示学习技术带来了重大挑战。目前，统一的3D模型较少，且现有的预训练方法无法同时有效处理物体级和场景级点云。本文提出了UniPre3D，这是首个能够无缝应用于任意尺度点云和任意架构3D模型的统一预训练方法。我们的方法以预测高斯基元作为预训练任务，并采用可微分的高斯溅射进行图像渲染，从而实现精确的像素级监督和端到端优化。为了进一步调控预训练任务的复杂性并引导模型关注几何结构，我们整合了预训练图像模型中的2D特征，以融入成熟的纹理知识。我们通过在各种物体级和场景级任务上的广泛实验，使用多种点云模型作为骨干网络，验证了所提出方法的普适有效性。代码可在https://github.com/wangzy22/UniPre3D获取。

English

The scale diversity of point cloud data presents significant challenges in developing unified representation learning techniques for 3D vision. Currently, there are few unified 3D models, and no existing pre-training method is equally effective for both object- and scene-level point clouds. In this paper, we introduce UniPre3D, the first unified pre-training method that can be seamlessly applied to point clouds of any scale and 3D models of any architecture. Our approach predicts Gaussian primitives as the pre-training task and employs differentiable Gaussian splatting to render images, enabling precise pixel-level supervision and end-to-end optimization. To further regulate the complexity of the pre-training task and direct the model's focus toward geometric structures, we integrate 2D features from pre-trained image models to incorporate well-established texture knowledge. We validate the universal effectiveness of our proposed method through extensive experiments across a variety of object- and scene-level tasks, using diverse point cloud models as backbones. Code is available at https://github.com/wangzy22/UniPre3D.

UniPre3D：基于跨模态高斯溅射的3D点云模型统一预训练

UniPre3D: Unified Pre-training of 3D Point Cloud Models with Cross-Modal Gaussian Splatting

摘要

Support