FruitNeRF：基於神經輝度場的水果計數統一框架

摘要

我們介紹了 FruitNeRF，一個統一的新型水果計數框架，利用最先進的視圖合成方法直接在3D中計算任何水果類型。我們的框架接受由單眼相機拍攝的無序姿勢圖像集，並在每個圖像中分割水果。為了使我們的系統獨立於水果類型，我們採用一個基礎模型，為任何水果生成二元分割遮罩。利用RGB和語義兩種模態，我們訓練了一個語義神經輻射場。通過對隱式 Fruit Field 進行均勻體積採樣，我們獲得僅包含水果的點雲。通過對提取的點雲應用級聚類，我們的方法實現了精確的水果計數。神經輻射場的使用相對於傳統方法（如物體跟踪或光流）提供了顯著的優勢，因為計數本身被提升到3D。我們的方法防止了對水果的重複計數，並避免計算無關的水果。我們使用真實世界和合成數據集來評估我們的方法。真實世界數據集包括三棵蘋果樹，具有手動計數的地面真相，一個具有一行和真實水果位置的基準蘋果數據集，而合成數據集包括各種水果類型，包括蘋果、李子、檸檬、梨、桃和芒果。此外，我們使用基礎模型與 U-Net 比較來評估水果計數的性能。

English

We introduce FruitNeRF, a unified novel fruit counting framework that leverages state-of-the-art view synthesis methods to count any fruit type directly in 3D. Our framework takes an unordered set of posed images captured by a monocular camera and segments fruit in each image. To make our system independent of the fruit type, we employ a foundation model that generates binary segmentation masks for any fruit. Utilizing both modalities, RGB and semantic, we train a semantic neural radiance field. Through uniform volume sampling of the implicit Fruit Field, we obtain fruit-only point clouds. By applying cascaded clustering on the extracted point cloud, our approach achieves precise fruit count.The use of neural radiance fields provides significant advantages over conventional methods such as object tracking or optical flow, as the counting itself is lifted into 3D. Our method prevents double counting fruit and avoids counting irrelevant fruit.We evaluate our methodology using both real-world and synthetic datasets. The real-world dataset consists of three apple trees with manually counted ground truths, a benchmark apple dataset with one row and ground truth fruit location, while the synthetic dataset comprises various fruit types including apple, plum, lemon, pear, peach, and mango.Additionally, we assess the performance of fruit counting using the foundation model compared to a U-Net.

FruitNeRF：基於神經輝度場的水果計數統一框架

FruitNeRF: A Unified Neural Radiance Field based Fruit Counting Framework

摘要

Summary

Support

Support