OlmoEarth：面向多模态地球观测的稳定潜在图像建模

摘要

地球观测数据呈现出独特的挑战性：它既具备图像的空间特性，又拥有视频或文本的序列特征，且具有高度多模态性。我们推出OlmoEarth——一个专为地球观测领域设计的多模态时空基础模型，其创新性地采用了自监督学习框架、掩码策略与损失函数。在与12种其他基础模型的多项研究基准及外部合作伙伴实际任务对比中，OlmoEarth实现了最先进的性能表现。在嵌入评估中，该模型在24项任务中的15项取得最佳性能；经全参数微调后，更在29项任务中的19项位列第一。我们将OlmoEarth部署为端到端平台的核心引擎，该平台集成了地球观测模型的数据采集、标注、训练与推理全流程。OlmoEarth平台将前沿基础模型与强大数据管理工具赋能给致力于解决全球重大问题的非营利组织与非政府机构。项目开源代码、训练数据及预训练权重已发布于https://github.com/allenai/olmoearth_pretrain。

English

Earth observation data presents a unique challenge: it is spatial like images, sequential like video or text, and highly multimodal. We present OlmoEarth: a multimodal, spatio-temporal foundation model that employs a novel self-supervised learning formulation, masking strategy, and loss all designed for the Earth observation domain. OlmoEarth achieves state-of-the-art performance compared to 12 other foundation models across a variety of research benchmarks and real-world tasks from external partners. When evaluating embeddings OlmoEarth achieves the best performance on 15 out of 24 tasks, and with full fine-tuning it is the best on 19 of 29 tasks. We deploy OlmoEarth as the backbone of an end-to-end platform for data collection, labeling, training, and inference of Earth observation models. The OlmoEarth Platform puts frontier foundation models and powerful data management tools into the hands of non-profits and NGOs working to solve the world's biggest problems. OlmoEarth source code, training data, and pre-trained weights are available at https://github.com/allenai/olmoearth_pretrain{https://github.com/allenai/olmoearth_pretrain}.

OlmoEarth：面向多模态地球观测的稳定潜在图像建模

OlmoEarth: Stable Latent Image Modeling for Multimodal Earth Observation

摘要

Support