UniverSat：用於地球觀測的分辨率與模態無關的Transformer

摘要

视觉Transformer（ViT）主导着计算机视觉领域。然而，它们对刚性补丁投影器的依赖阻碍了向地球观测（EO）的迁移，因为地球观测中的输入模态、尺度和分辨率差异极大。我们提出UniverSat，这是一种基于ViT风格的主干网络，其核心是通用补丁编码器（Universal Patch Encoder），能将来自任意空间、光谱和时间分辨率，以及光学和非光学传感器的补丁，通过一组共享权重映射到共享嵌入空间中。这使得我们能够通过自监督方式在异构多模态语料库上训练单一模型，从而获得鲁棒的、与传感器无关的空间特征。我们在GeoBench、PANGEABench和SpectralEarth等标准地球观测基准上进行的分类和分割任务中，验证了该方法的强劲性能。我们的代码和模型已开源在https://github.com/gastruc/UniverSat。

English

Vision Transformers (ViT) dominate computer vision. However, their reliance on rigid patch projectors hinders transfer to Earth Observation (EO), where input modalities, scales, and resolutions vary widely. We introduce UniverSat, a ViT-style backbone built around a Universal Patch Encoder that maps patches from arbitrary spatial, spectral, and temporal resolutions, and from both optical and non-optical sensors, into a shared embedding space with a shared set of weights. This enables training a single model on heterogeneous multimodal corpora via self-supervision, yielding robust, sensor-agnostic spatial features. We validate this approach with strong results across classification and segmentation on standard EO benchmarks from GeoBench, PANGEABench, and SpectralEarth. Our code and models are available at https://github.com/gastruc/UniverSat.