UniverSat: 面向地球观测的分辨率与模态无关的Transformer

摘要

视觉Transformer（ViT）主导着计算机视觉领域。然而，它们对刚性分块投影器的依赖阻碍了向地球观测（EO）任务的迁移，因为EO的输入模态、尺度和分辨率变化极大。我们提出UniverSat，一种基于通用分块编码器（Universal Patch Encoder）的ViT风格骨干网络，该编码器能将来自任意空间、光谱和时间分辨率、以及光学与非光学传感器的分块，通过共享权重映射到统一的嵌入空间。这使得我们能够以自监督方式在异构多模态数据集上训练单一模型，从而获得稳健且与传感器无关的空间特征。我们通过GeoBench、PANGEABench和SpectralEarth等标准EO基准上的分类与分割任务验证了该方法的显著效果。代码与模型已开源至 https://github.com/gastruc/UniverSat。

English

Vision Transformers (ViT) dominate computer vision. However, their reliance on rigid patch projectors hinders transfer to Earth Observation (EO), where input modalities, scales, and resolutions vary widely. We introduce UniverSat, a ViT-style backbone built around a Universal Patch Encoder that maps patches from arbitrary spatial, spectral, and temporal resolutions, and from both optical and non-optical sensors, into a shared embedding space with a shared set of weights. This enables training a single model on heterogeneous multimodal corpora via self-supervision, yielding robust, sensor-agnostic spatial features. We validate this approach with strong results across classification and segmentation on standard EO benchmarks from GeoBench, PANGEABench, and SpectralEarth. Our code and models are available at https://github.com/gastruc/UniverSat.