图像喷溅：超快速单视图三维重建

摘要

我们介绍了Splatter Image，这是一种在38 FPS运行的单目3D物体重建的超快速方法。Splatter Image基于高斯飞溅技术，该技术最近为多视角重建带来了实时渲染、快速训练和出色的扩展性。我们首次将高斯飞溅技术应用于单目重建设置中。我们的方法是基于学习的，在测试时，重建仅需要神经网络的前向评估。Splatter Image的主要创新在于其惊人简单的设计：它使用2D图像到图像的网络，将输入图像映射到每个像素一个3D高斯。因此得到的高斯具有图像的形式，即Splatter Image。我们进一步扩展了该方法，通过添加跨视图注意力，使其能够处理多于一个图像的输入。由于渲染器的速度（588 FPS）很快，我们可以在训练时仅使用单个GPU，同时在每次迭代中生成整个图像，以优化像LPIPS这样的感知度量。在标准基准测试中，我们不仅展示了快速重建，而且在PSNR、LPIPS和其他指标方面取得了比最近和更昂贵的基线更好的结果。

English

We introduce the Splatter Image, an ultra-fast approach for monocular 3D object reconstruction which operates at 38 FPS. Splatter Image is based on Gaussian Splatting, which has recently brought real-time rendering, fast training, and excellent scaling to multi-view reconstruction. For the first time, we apply Gaussian Splatting in a monocular reconstruction setting. Our approach is learning-based, and, at test time, reconstruction only requires the feed-forward evaluation of a neural network. The main innovation of Splatter Image is the surprisingly straightforward design: it uses a 2D image-to-image network to map the input image to one 3D Gaussian per pixel. The resulting Gaussians thus have the form of an image, the Splatter Image. We further extend the method to incorporate more than one image as input, which we do by adding cross-view attention. Owning to the speed of the renderer (588 FPS), we can use a single GPU for training while generating entire images at each iteration in order to optimize perceptual metrics like LPIPS. On standard benchmarks, we demonstrate not only fast reconstruction but also better results than recent and much more expensive baselines in terms of PSNR, LPIPS, and other metrics.

图像喷溅：超快速单视图三维重建

Splatter Image: Ultra-Fast Single-View 3D Reconstruction

摘要

Support