树木园：一个支持生物多样性人工智能的大型多模态数据集

Arboretum: A Large Multimodal Dataset Enabling AI for Biodiversity

June 25, 2024

作者: Chih-Hsuan Yang, Benjamin Feuer, Zaki Jubery, Zi K. Deng, Andre Nakkab, Md Zahid Hasan, Shivani Chiranjeevi, Kelly Marshall, Nirmal Baishnab, Asheesh K Singh, Arti Singh, Soumik Sarkar, Nirav Merchant, Chinmay Hegde, Baskar Ganapathysubramanian

cs.AI

摘要

我们介绍了Arboretum，这是一个旨在推动生物多样性应用人工智能发展的最大公开可访问数据集。该数据集从iNaturalist社区科学平台精心筛选，经领域专家审核以确保准确性，包含了1.346亿张图片，比现有数据集规模大一个数量级。该数据集涵盖了来自鸟类（Aves）、蜘蛛/蜱/螨（Arachnida）、昆虫（Insecta）、植物（Plantae）、真菌/蘑菇（Fungi）、蜗牛（Mollusca）以及蛇类/蜥蜴（Reptilia）等多样物种的图像-语言配对数据，为生物多样性评估和农业研究的多模态视觉-语言人工智能模型提供了宝贵资源。每张图片都标注了科学名称、分类学细节和通用名称，增强了人工智能模型训练的稳健性。 我们展示了Arboretum的价值，发布了一套使用4000万张带字幕图片子集训练的CLIP模型。我们引入了几个新的严格评估基准，报告了零样本学习的准确性，并评估了各个生命周期阶段、稀有物种、混淆物种以及不同分类层次的准确性。 我们预计Arboretum将推动开发能够实现从害虫控制策略、作物监测，到全球生物多样性评估和环境保护等各种数字工具的人工智能模型。这些进展对于确保粮食安全、保护生态系统以及减缓气候变化影响至关重要。Arboretum是公开可用、易于访问且可立即使用的。 请访问https://baskargroup.github.io/Arboretum/（项目网站）获取我们的数据、模型和代码链接。

English

We introduce Arboretum, the largest publicly accessible dataset designed to advance AI for biodiversity applications. This dataset, curated from the iNaturalist community science platform and vetted by domain experts to ensure accuracy, includes 134.6 million images, surpassing existing datasets in scale by an order of magnitude. The dataset encompasses image-language paired data for a diverse set of species from birds (Aves), spiders/ticks/mites (Arachnida), insects (Insecta), plants (Plantae), fungus/mushrooms (Fungi), snails (Mollusca), and snakes/lizards (Reptilia), making it a valuable resource for multimodal vision-language AI models for biodiversity assessment and agriculture research. Each image is annotated with scientific names, taxonomic details, and common names, enhancing the robustness of AI model training. We showcase the value of Arboretum by releasing a suite of CLIP models trained using a subset of 40 million captioned images. We introduce several new benchmarks for rigorous assessment, report accuracy for zero-shot learning, and evaluations across life stages, rare species, confounding species, and various levels of the taxonomic hierarchy. We anticipate that Arboretum will spur the development of AI models that can enable a variety of digital tools ranging from pest control strategies, crop monitoring, and worldwide biodiversity assessment and environmental conservation. These advancements are critical for ensuring food security, preserving ecosystems, and mitigating the impacts of climate change. Arboretum is publicly available, easily accessible, and ready for immediate use. Please see the https://baskargroup.github.io/Arboretum/{project website} for links to our data, models, and code.