高效X射线新视角合成的辐射高斯飞溅

摘要

由于X射线的穿透能力强于自然光，因此广泛应用于传输成像。在渲染新视角的X射线投影时，现有方法主要基于NeRF，但存在训练时间长和推理速度慢的问题。本文提出了一种基于3D高斯飞溅的框架，名为X-Gaussian，用于X射线新视角合成。首先，我们重新设计了一个受X射线成像各向同性特性启发的辐射高斯点云模型。我们的模型在学习预测3D点的辐射强度时排除了视角方向的影响。基于这一模型，我们开发了一个带CUDA实现的可微辐射光栅化（DRR）。其次，我们定制了一个角度-姿态立方体均匀初始化（ACUI）策略，直接使用X射线扫描仪的参数计算相机信息，然后在包围扫描对象的立方体内均匀采样点位置。实验表明，我们的X-Gaussian在享受少于15%的训练时间和超过73倍推理速度的同时，优于最先进的方法6.5 dB。在稀疏视图CT重建上的应用也揭示了我们方法的实际价值。代码和模型将在https://github.com/caiyuanhao1998/X-Gaussian 上公开。训练过程可视化的视频演示请访问https://www.youtube.com/watch?v=gDVf_Ngeghg。

English

X-ray is widely applied for transmission imaging due to its stronger penetration than natural light. When rendering novel view X-ray projections, existing methods mainly based on NeRF suffer from long training time and slow inference speed. In this paper, we propose a 3D Gaussian splatting-based framework, namely X-Gaussian, for X-ray novel view synthesis. Firstly, we redesign a radiative Gaussian point cloud model inspired by the isotropic nature of X-ray imaging. Our model excludes the influence of view direction when learning to predict the radiation intensity of 3D points. Based on this model, we develop a Differentiable Radiative Rasterization (DRR) with CUDA implementation. Secondly, we customize an Angle-pose Cuboid Uniform Initialization (ACUI) strategy that directly uses the parameters of the X-ray scanner to compute the camera information and then uniformly samples point positions within a cuboid enclosing the scanned object. Experiments show that our X-Gaussian outperforms state-of-the-art methods by 6.5 dB while enjoying less than 15% training time and over 73x inference speed. The application on sparse-view CT reconstruction also reveals the practical values of our method. Code and models will be publicly available at https://github.com/caiyuanhao1998/X-Gaussian . A video demo of the training process visualization is at https://www.youtube.com/watch?v=gDVf_Ngeghg .

高效X射线新视角合成的辐射高斯飞溅

Radiative Gaussian Splatting for Efficient X-ray Novel View Synthesis

摘要

Support