NeuFlow：在机器人上使用边缘设备进行实时、高精度的光流估计

NeuFlow: Real-time, High-accuracy Optical Flow Estimation on Robots Using Edge Devices

March 15, 2024

作者: Zhiyong Zhang, Huaizu Jiang, Hanumant Singh

cs.AI

摘要

实时高精度光流估计是各种应用中的关键组成部分，包括机器人定位和建图、目标跟踪以及计算机视觉中的活动识别。尽管最近基于学习的光流方法取得了高精度，但通常伴随着沉重的计算成本。本文提出了一种名为NeuFlow的高效光流架构，旨在解决高精度和计算成本问题。该架构采用了全局到局部的方案。通过提取不同空间分辨率下输入图像的特征，我们首先使用全局匹配在1/16分辨率上估计初始光流，捕捉大位移，然后在1/8分辨率上通过轻量级CNN层进行优化以获得更好的精度。我们在Jetson Orin Nano和RTX 2080上评估了我们的方法，展示了在不同计算平台上的效率改进。与几种最先进的方法相比，我们实现了显著的10倍至80倍加速，同时保持可比较的精度。我们的方法在边缘计算平台上实现约30帧每秒的性能，这在部署诸如无人机等小型机器人上的复杂计算机视觉任务方面具有重大突破。完整的训练和评估代码可在https://github.com/neufieldrobotics/NeuFlow获取。

English

Real-time high-accuracy optical flow estimation is a crucial component in various applications, including localization and mapping in robotics, object tracking, and activity recognition in computer vision. While recent learning-based optical flow methods have achieved high accuracy, they often come with heavy computation costs. In this paper, we propose a highly efficient optical flow architecture, called NeuFlow, that addresses both high accuracy and computational cost concerns. The architecture follows a global-to-local scheme. Given the features of the input images extracted at different spatial resolutions, global matching is employed to estimate an initial optical flow on the 1/16 resolution, capturing large displacement, which is then refined on the 1/8 resolution with lightweight CNN layers for better accuracy. We evaluate our approach on Jetson Orin Nano and RTX 2080 to demonstrate efficiency improvements across different computing platforms. We achieve a notable 10x-80x speedup compared to several state-of-the-art methods, while maintaining comparable accuracy. Our approach achieves around 30 FPS on edge computing platforms, which represents a significant breakthrough in deploying complex computer vision tasks such as SLAM on small robots like drones. The full training and evaluation code is available at https://github.com/neufieldrobotics/NeuFlow.