ThermalGen：基于风格解耦流模型的RGB-热成像图像转换生成网络
ThermalGen: Style-Disentangled Flow-Based Generative Models for RGB-to-Thermal Image Translation
September 29, 2025
作者: Jiuhong Xiao, Roshan Nayak, Ning Zhang, Daniel Tortei, Giuseppe Loianno
cs.AI
摘要
配对的RGB-热成像数据对于视觉-热成像传感器融合及跨模态任务至关重要，这些任务包括多模态图像对齐与检索等重要应用。然而，同步且校准的RGB-热成像图像对的稀缺，严重阻碍了这些领域的进展。为应对这一挑战，RGB到热成像（RGB-T）图像翻译技术应运而生，它能够从丰富的RGB数据集中合成热成像图像，用于训练目的。本研究提出ThermalGen，一种基于自适应流的RGB-T图像翻译生成模型，融合了RGB图像条件架构与风格解耦机制。为支持大规模训练，我们整合了八个公开的卫星-航空、航空及地面RGB-T配对数据集，并引入了三个新的大规模卫星-航空RGB-T数据集——DJI-day、Bosonplus-day和Bosonplus-night，这些数据集跨越了不同时间、传感器类型及地理区域。在多个RGB-T基准上的广泛评估表明，ThermalGen在翻译性能上可与现有的基于GAN和扩散模型的方法相媲美甚至更优。据我们所知，ThermalGen是首个能够合成反映显著视角变化、传感器特性及环境条件差异的热成像图像的RGB-T图像翻译模型。项目页面：http://xjh19971.github.io/ThermalGen
English
Paired RGB-thermal data is crucial for visual-thermal sensor fusion and
cross-modality tasks, including important applications such as multi-modal
image alignment and retrieval. However, the scarcity of synchronized and
calibrated RGB-thermal image pairs presents a major obstacle to progress in
these areas. To overcome this challenge, RGB-to-Thermal (RGB-T) image
translation has emerged as a promising solution, enabling the synthesis of
thermal images from abundant RGB datasets for training purposes. In this study,
we propose ThermalGen, an adaptive flow-based generative model for RGB-T image
translation, incorporating an RGB image conditioning architecture and a
style-disentangled mechanism. To support large-scale training, we curated eight
public satellite-aerial, aerial, and ground RGB-T paired datasets, and
introduced three new large-scale satellite-aerial RGB-T datasets--DJI-day,
Bosonplus-day, and Bosonplus-night--captured across diverse times, sensor
types, and geographic regions. Extensive evaluations across multiple RGB-T
benchmarks demonstrate that ThermalGen achieves comparable or superior
translation performance compared to existing GAN-based and diffusion-based
methods. To our knowledge, ThermalGen is the first RGB-T image translation
model capable of synthesizing thermal images that reflect significant
variations in viewpoints, sensor characteristics, and environmental conditions.
Project page: http://xjh19971.github.io/ThermalGen