NeuFlow：在機器人上使用邊緣裝置進行實時、高精度的光流估計

NeuFlow: Real-time, High-accuracy Optical Flow Estimation on Robots Using Edge Devices

March 15, 2024

作者: Zhiyong Zhang, Huaizu Jiang, Hanumant Singh

cs.AI

摘要

實時高精度光流估計在各種應用中至關重要，包括機器人定位和地圖繪製、物體追蹤以及計算機視覺中的活動識別。儘管最近基於學習的光流方法取得了高精度，但通常伴隨著沉重的計算成本。本文提出了一種名為NeuFlow的高效光流架構，旨在解決高精度和計算成本問題。該架構採用全局到局部的方案。根據提取自不同空間分辨率的輸入圖像特徵，我們採用全局匹配來估計1/16分辨率上的初始光流，捕捉大位移，然後在1/8分辨率上使用輕量級CNN層進行更好的精度調整。我們在Jetson Orin Nano和RTX 2080上評估我們的方法，以展示在不同計算平台上的效率改進。與幾種最先進的方法相比，我們實現了顯著的10倍至80倍加速，同時保持可比的精度。我們的方法在邊緣計算平台上實現約30 FPS，這在部署複雜的計算機視覺任務（如SLAM）到像無人機等小型機器人上具有重大突破。完整的訓練和評估代碼可在https://github.com/neufieldrobotics/NeuFlow 上找到。

English

Real-time high-accuracy optical flow estimation is a crucial component in various applications, including localization and mapping in robotics, object tracking, and activity recognition in computer vision. While recent learning-based optical flow methods have achieved high accuracy, they often come with heavy computation costs. In this paper, we propose a highly efficient optical flow architecture, called NeuFlow, that addresses both high accuracy and computational cost concerns. The architecture follows a global-to-local scheme. Given the features of the input images extracted at different spatial resolutions, global matching is employed to estimate an initial optical flow on the 1/16 resolution, capturing large displacement, which is then refined on the 1/8 resolution with lightweight CNN layers for better accuracy. We evaluate our approach on Jetson Orin Nano and RTX 2080 to demonstrate efficiency improvements across different computing platforms. We achieve a notable 10x-80x speedup compared to several state-of-the-art methods, while maintaining comparable accuracy. Our approach achieves around 30 FPS on edge computing platforms, which represents a significant breakthrough in deploying complex computer vision tasks such as SLAM on small robots like drones. The full training and evaluation code is available at https://github.com/neufieldrobotics/NeuFlow.