ThermalNeRF：热辐射场

ThermalNeRF: Thermal Radiance Fields

July 22, 2024

作者: Yvette Y. Lin, Xin-Yi Pan, Sara Fridovich-Keil, Gordon Wetzstein

cs.AI

摘要

热成像具有各种应用，从农业监测到建筑检查，再到在低光、雾和雨等恶劣条件下成像。然而，由于长波红外（LWIR）图像中分辨率相对较低且特征有限，重建3D热场景存在一些挑战。为了克服这些挑战，我们提出了一个统一的框架，用于从一组LWIR和RGB图像中重建场景，利用多光谱辐射场来表示可见光和红外相机共同观察的场景，从而利用两个光谱间的信息。我们通过使用简单的校准目标，对RGB和红外相机进行彼此校准作为预处理步骤。我们在从手持热成像相机拍摄的真实RGB和LWIR照片集上展示了我们的方法，展示了我们的方法在可见光和红外光谱中场景表示方面的有效性。我们展示了我们的方法能够进行热超分辨率，并在视觉上消除障碍物，揭示在RGB或热通道中被遮挡的物体。请参阅https://yvette256.github.io/thermalnerf 查看视频结果以及我们的代码和数据集发布。

English

Thermal imaging has a variety of applications, from agricultural monitoring to building inspection to imaging under poor visibility, such as in low light, fog, and rain. However, reconstructing thermal scenes in 3D presents several challenges due to the comparatively lower resolution and limited features present in long-wave infrared (LWIR) images. To overcome these challenges, we propose a unified framework for scene reconstruction from a set of LWIR and RGB images, using a multispectral radiance field to represent a scene viewed by both visible and infrared cameras, thus leveraging information across both spectra. We calibrate the RGB and infrared cameras with respect to each other, as a preprocessing step using a simple calibration target. We demonstrate our method on real-world sets of RGB and LWIR photographs captured from a handheld thermal camera, showing the effectiveness of our method at scene representation across the visible and infrared spectra. We show that our method is capable of thermal super-resolution, as well as visually removing obstacles to reveal objects that are occluded in either the RGB or thermal channels. Please see https://yvette256.github.io/thermalnerf for video results as well as our code and dataset release.