ThermalNeRF：熱輻射場

ThermalNeRF: Thermal Radiance Fields

July 22, 2024

作者: Yvette Y. Lin, Xin-Yi Pan, Sara Fridovich-Keil, Gordon Wetzstein

cs.AI

摘要

熱成像技術具有各種應用，從農業監測到建築檢查，再到在低光、霧、雨等能見度差的環境下進行成像。然而，由於長波紅外線（LWIR）圖像中的分辨率相對較低且特徵有限，因此在3D中重建熱場景存在幾個挑戰。為了克服這些挑戰，我們提出了一個統一的框架，用於從一組LWIR和RGB圖像中重建場景，使用多光譜輻射場來表示可見光和紅外攝像機都觀察到的場景，從而利用兩個光譜間的信息。我們對RGB和紅外攝像機進行校準，作為預處理步驟，使用簡單的校準目標。我們展示了我們的方法應用於從手持熱成像相機拍攝的真實RGB和LWIR照片集，展示了我們的方法在可見光和紅外光譜間的場景表示效果。我們展示了我們的方法能夠進行熱超分辨率，以及在視覺上消除障礙物，揭示在RGB或熱通道中被遮擋的物體。請參見https://yvette256.github.io/thermalnerf 以查看視頻結果以及我們的代碼和數據集。

English

Thermal imaging has a variety of applications, from agricultural monitoring to building inspection to imaging under poor visibility, such as in low light, fog, and rain. However, reconstructing thermal scenes in 3D presents several challenges due to the comparatively lower resolution and limited features present in long-wave infrared (LWIR) images. To overcome these challenges, we propose a unified framework for scene reconstruction from a set of LWIR and RGB images, using a multispectral radiance field to represent a scene viewed by both visible and infrared cameras, thus leveraging information across both spectra. We calibrate the RGB and infrared cameras with respect to each other, as a preprocessing step using a simple calibration target. We demonstrate our method on real-world sets of RGB and LWIR photographs captured from a handheld thermal camera, showing the effectiveness of our method at scene representation across the visible and infrared spectra. We show that our method is capable of thermal super-resolution, as well as visually removing obstacles to reveal objects that are occluded in either the RGB or thermal channels. Please see https://yvette256.github.io/thermalnerf for video results as well as our code and dataset release.