邁向統一的哥白尼地球視覺基礎模型

摘要

地球觀測（EO）基礎模型的進展，已釋放了利用大規模衛星數據從太空學習通用表徵的潛力，這對我們星球至關重要的廣泛下游應用大有裨益。然而，現有大多數努力仍局限於固定光譜傳感器，僅聚焦於地球表面，且忽視了影像之外有價值的元數據。在本研究中，我們朝著下一代EO基礎模型邁進，提出了三大關鍵組成部分：1）Copernicus-Pretrain，一個大規模預訓練數據集，整合了來自所有主要哥白尼哨兵任務的1870萬張對齊圖像，覆蓋從地球表面到大氣層的範圍；2）Copernicus-FM，一個統一的基礎模型，能夠通過擴展的動態超網絡和靈活的元數據編碼，處理任何光譜或非光譜傳感器模態；以及3）Copernicus-Bench，一個系統化的評估基準，包含從預處理到針對每項哨兵任務的專業應用共15個層次化的下游任務。我們的數據集、模型和基準極大地提升了EO基礎模型的可擴展性、多功能性及多模態適應能力，同時也為連接EO、天氣和氣候研究開闢了新的機遇。代碼、數據集和模型可在https://github.com/zhu-xlab/Copernicus-FM獲取。

English

Advances in Earth observation (EO) foundation models have unlocked the potential of big satellite data to learn generic representations from space, benefiting a wide range of downstream applications crucial to our planet. However, most existing efforts remain limited to fixed spectral sensors, focus solely on the Earth's surface, and overlook valuable metadata beyond imagery. In this work, we take a step towards next-generation EO foundation models with three key components: 1) Copernicus-Pretrain, a massive-scale pretraining dataset that integrates 18.7M aligned images from all major Copernicus Sentinel missions, spanning from the Earth's surface to its atmosphere; 2) Copernicus-FM, a unified foundation model capable of processing any spectral or non-spectral sensor modality using extended dynamic hypernetworks and flexible metadata encoding; and 3) Copernicus-Bench, a systematic evaluation benchmark with 15 hierarchical downstream tasks ranging from preprocessing to specialized applications for each Sentinel mission. Our dataset, model, and benchmark greatly improve the scalability, versatility, and multimodal adaptability of EO foundation models, while also creating new opportunities to connect EO, weather, and climate research. Codes, datasets and models are available at https://github.com/zhu-xlab/Copernicus-FM.

邁向統一的哥白尼地球視覺基礎模型

Towards a Unified Copernicus Foundation Model for Earth Vision

摘要

Summary

Support

Support