NerfBridge：将实时在线神经辐射场训练引入机器人技术

摘要

本文发表在2023年IEEE国际机器人与自动化会议的非传统空间表示研讨会上。神经辐射场（NeRFs）是一类隐式场景表示，可以从彩色图像中对3D环境进行建模。NeRFs具有表现力，可以对真实世界环境的复杂和多尺度几何进行建模，这使它们潜在地成为机器人应用的强大工具。现代NeRF训练库可以在几秒钟内从静态数据集生成逼真的NeRF，但它们设计用于离线使用，并需要一个缓慢的姿势优化预计算步骤。在这项工作中，我们提出了NerfBridge，这是一个开源桥梁，连接了机器人操作系统（ROS）和流行的Nerfstudio库，用于从图像流实时在线训练NeRF。NerfBridge通过提供一个可扩展的接口，连接了Nerfstudio提供的高效训练流水线和模型库，从而促进了在机器人学中应用NeRF的研究的快速发展。作为示例用例，我们概述了一个硬件设置，可以使用NerfBridge来训练一个NeRF，该NeRF是通过安装在四旋翼飞行器上的摄像头捕获的室内和室外环境的图像。附带视频链接https://youtu.be/EH0SLn-RcDg 和代码链接https://github.com/javieryu/nerf_bridge。

English

This work was presented at the IEEE International Conference on Robotics and Automation 2023 Workshop on Unconventional Spatial Representations. Neural radiance fields (NeRFs) are a class of implicit scene representations that model 3D environments from color images. NeRFs are expressive, and can model the complex and multi-scale geometry of real world environments, which potentially makes them a powerful tool for robotics applications. Modern NeRF training libraries can generate a photo-realistic NeRF from a static data set in just a few seconds, but are designed for offline use and require a slow pose optimization pre-computation step. In this work we propose NerfBridge, an open-source bridge between the Robot Operating System (ROS) and the popular Nerfstudio library for real-time, online training of NeRFs from a stream of images. NerfBridge enables rapid development of research on applications of NeRFs in robotics by providing an extensible interface to the efficient training pipelines and model libraries provided by Nerfstudio. As an example use case we outline a hardware setup that can be used NerfBridge to train a NeRF from images captured by a camera mounted to a quadrotor in both indoor and outdoor environments. For accompanying video https://youtu.be/EH0SLn-RcDg and code https://github.com/javieryu/nerf_bridge.

NerfBridge：将实时在线神经辐射场训练引入机器人技术

NerfBridge: Bringing Real-time, Online Neural Radiance Field Training to Robotics

摘要

Support