FruitNeRF：基于神经辐射场的水果计数统一框架

摘要

我们介绍了FruitNeRF，这是一个统一的新型水果计数框架，利用最先进的视图合成方法直接在3D中计数任何类型的水果。我们的框架接收由单目摄像头捕获的无序姿态图像集，并在每个图像中分割水果。为了使我们的系统独立于水果类型，我们采用一个基础模型，为任何水果生成二进制分割掩模。利用RGB和语义两种模态，我们训练了一个语义神经辐射场。通过对隐式水果场进行均匀体积采样，我们获得仅包含水果的点云。通过在提取的点云上应用级联聚类，我们的方法实现了精确的水果计数。使用神经辐射场相比传统方法（如目标跟踪或光流）提供了显著优势，因为计数本身被提升到了3D。我们的方法可以防止水果被重复计数，并避免计数无关的水果。我们使用真实世界和合成数据集评估了我们的方法论。真实世界数据集包括三棵苹果树，具有手动计数的地面真值，一个具有一行和地面真实水果位置的基准苹果数据集，而合成数据集包括各种水果类型，包括苹果、李子、柠檬、梨、桃子和芒果。此外，我们评估了使用基础模型与U-Net相比进行水果计数的性能。

English

We introduce FruitNeRF, a unified novel fruit counting framework that leverages state-of-the-art view synthesis methods to count any fruit type directly in 3D. Our framework takes an unordered set of posed images captured by a monocular camera and segments fruit in each image. To make our system independent of the fruit type, we employ a foundation model that generates binary segmentation masks for any fruit. Utilizing both modalities, RGB and semantic, we train a semantic neural radiance field. Through uniform volume sampling of the implicit Fruit Field, we obtain fruit-only point clouds. By applying cascaded clustering on the extracted point cloud, our approach achieves precise fruit count.The use of neural radiance fields provides significant advantages over conventional methods such as object tracking or optical flow, as the counting itself is lifted into 3D. Our method prevents double counting fruit and avoids counting irrelevant fruit.We evaluate our methodology using both real-world and synthetic datasets. The real-world dataset consists of three apple trees with manually counted ground truths, a benchmark apple dataset with one row and ground truth fruit location, while the synthetic dataset comprises various fruit types including apple, plum, lemon, pear, peach, and mango.Additionally, we assess the performance of fruit counting using the foundation model compared to a U-Net.

FruitNeRF：基于神经辐射场的水果计数统一框架

FruitNeRF: A Unified Neural Radiance Field based Fruit Counting Framework

摘要

Summary

Support

Support