迈向统一的哥白尼地球视觉基础模型

摘要

地球观测（EO）基础模型的进展释放了大规模卫星数据的潜力，使其能够从太空中学习通用表征，从而惠及对地球至关重要的广泛下游应用。然而，现有研究大多局限于固定光谱传感器，仅关注地球表面，并忽视了图像之外的有价值元数据。在本研究中，我们朝着下一代EO基础模型迈出了重要一步，提出了三个核心组件：1）Copernicus-Pretrain，一个大规模预训练数据集，整合了来自所有主要哥白尼哨兵任务的1870万张对齐图像，覆盖从地球表面到大气层的全方位观测；2）Copernicus-FM，一个统一的基础模型，通过扩展的动态超网络和灵活的元数据编码，能够处理任何光谱或非光谱传感器模态；3）Copernics-Bench，一个系统化的评估基准，包含15个层次化的下游任务，从预处理到各哨兵任务的专业应用。我们的数据集、模型和基准显著提升了EO基础模型的可扩展性、多功能性和多模态适应性，同时为连接EO、天气和气候研究开辟了新的机遇。代码、数据集和模型可在https://github.com/zhu-xlab/Copernicus-FM获取。

English

Advances in Earth observation (EO) foundation models have unlocked the potential of big satellite data to learn generic representations from space, benefiting a wide range of downstream applications crucial to our planet. However, most existing efforts remain limited to fixed spectral sensors, focus solely on the Earth's surface, and overlook valuable metadata beyond imagery. In this work, we take a step towards next-generation EO foundation models with three key components: 1) Copernicus-Pretrain, a massive-scale pretraining dataset that integrates 18.7M aligned images from all major Copernicus Sentinel missions, spanning from the Earth's surface to its atmosphere; 2) Copernicus-FM, a unified foundation model capable of processing any spectral or non-spectral sensor modality using extended dynamic hypernetworks and flexible metadata encoding; and 3) Copernicus-Bench, a systematic evaluation benchmark with 15 hierarchical downstream tasks ranging from preprocessing to specialized applications for each Sentinel mission. Our dataset, model, and benchmark greatly improve the scalability, versatility, and multimodal adaptability of EO foundation models, while also creating new opportunities to connect EO, weather, and climate research. Codes, datasets and models are available at https://github.com/zhu-xlab/Copernicus-FM.

迈向统一的哥白尼地球视觉基础模型

Towards a Unified Copernicus Foundation Model for Earth Vision

摘要

Summary

Support

Support