用於高效X射線新視角合成的輻射高斯飛濺

摘要

X射線由於比自然光具有更強的穿透力，因此廣泛應用於傳輸成像。在渲染新視角的X射線投影時，現有方法主要基於NeRF，但存在訓練時間長和推斷速度慢的問題。本文提出了一種基於3D高斯飛濺的框架，名為X-Gaussian，用於X射線新視角合成。首先，我們重新設計了一個受X射線成像各向同性特性啟發的輻射高斯點雲模型。我們的模型在學習預測3D點的輻射強度時排除了視角方向的影響。基於此模型，我們開發了一種可微的輻射光柵化（DRR），並實現了CUDA版本。其次，我們定制了一種角度-姿勢立方體均勻初始化（ACUI）策略，直接使用X射線掃描儀的參數來計算相機信息，然後在包圍被掃描物體的立方體內均勻採樣點位置。實驗表明，我們的X-Gaussian在享受不到15%的訓練時間和超過73倍的推斷速度的情況下，優於最先進的方法6.5 dB。在稀疏視圖CT重建上的應用也揭示了我們方法的實際價值。代碼和模型將在https://github.com/caiyuanhao1998/X-Gaussian 上公開。訓練過程可視化的視頻演示在https://www.youtube.com/watch?v=gDVf_Ngeghg。

English

X-ray is widely applied for transmission imaging due to its stronger penetration than natural light. When rendering novel view X-ray projections, existing methods mainly based on NeRF suffer from long training time and slow inference speed. In this paper, we propose a 3D Gaussian splatting-based framework, namely X-Gaussian, for X-ray novel view synthesis. Firstly, we redesign a radiative Gaussian point cloud model inspired by the isotropic nature of X-ray imaging. Our model excludes the influence of view direction when learning to predict the radiation intensity of 3D points. Based on this model, we develop a Differentiable Radiative Rasterization (DRR) with CUDA implementation. Secondly, we customize an Angle-pose Cuboid Uniform Initialization (ACUI) strategy that directly uses the parameters of the X-ray scanner to compute the camera information and then uniformly samples point positions within a cuboid enclosing the scanned object. Experiments show that our X-Gaussian outperforms state-of-the-art methods by 6.5 dB while enjoying less than 15% training time and over 73x inference speed. The application on sparse-view CT reconstruction also reveals the practical values of our method. Code and models will be publicly available at https://github.com/caiyuanhao1998/X-Gaussian . A video demo of the training process visualization is at https://www.youtube.com/watch?v=gDVf_Ngeghg .

用於高效X射線新視角合成的輻射高斯飛濺

Radiative Gaussian Splatting for Efficient X-ray Novel View Synthesis

摘要

Support