PointDiT：用於單目幾何估計的像素空間擴散

摘要

当前最先进的单张图像三维重建方法通常依赖复杂的混合架构与损失函数，或将几何形状压缩至潜在空间以利用预训练的潜在扩散模型。本研究表明，此类架构上的冗余与复杂的损失函数设计并非必要。我们提出一种极简的像素空间扩散Transformer，基于普通ViT构建，直接对原始三维点图块进行操作，并以预训练DINOv3的图像标记为条件。与现有的潜在扩散方法不同，我们从头训练扩散骨干网络，无需点图分词器。尽管设计简洁，该方法性能超越复杂的潜在扩散模型，同时显著简化了混合方案。值得注意的是，它能生成更锐利的几何结构，并在高模糊区域（如透明物体）中更具鲁棒性。

English

State-of-the-art single-image 3D reconstruction methods often rely on complex hybrid architectures and loss functions, or compress geometry into latent spaces in order to leverage pre-trained latent diffusion models. In this work, we show that such architectural overhead and intricate loss formulations are unnecessary. We introduce a minimalist pixel-space Diffusion Transformer, built on a plain ViT, that operates directly on raw 3D point map patches and is conditioned on image tokens from a pre-trained DINOv3. Unlike existing latent diffusion approaches, we train our diffusion backbone entirely from scratch, eliminating the need for point map tokenizers. Despite its simplicity, our approach surpasses complex latent-based diffusion models while remaining significantly simpler than hybrid alternatives. Notably, it produces sharper geometric structure and is more robust in highly ambiguous regions, such as transparent objects.