NerfBridge：將實時、在線神經輻射場訓練引入機器人技術

摘要

本研究是在2023年IEEE國際機器人與自動化大會上的非傳統空間表示研討會上發表的。神經輻射場（NeRFs）是一類隱式場景表示，從彩色圖像中建模3D環境。NeRFs具有表現力，能夠模擬真實世界環境的複雜和多尺度幾何結構，這使它們潛在地成為機器人應用的強大工具。現代NeRF訓練庫可以在幾秒內從靜態數據集生成逼真的NeRF，但設計用於離線使用，需要緩慢的姿態優化預計算步驟。在這項工作中，我們提出了NerfBridge，這是一個開源橋樑，連接機器人操作系統（ROS）和流行的Nerfstudio庫，用於從圖像流中實時、在線訓練NeRF。NerfBridge通過提供一個可擴展的接口，連接到Nerfstudio提供的高效訓練管線和模型庫，從而促進了在機器人學中應用NeRF的研究的快速發展。作為一個示例用例，我們概述了一個硬體設置，可以使用NerfBridge來訓練一個NeRF，該NeRF是通過安裝在四旋翼機上的相機捕捉室內和室外環境的圖像而獲得的。有關相應視頻，請參見https://youtu.be/EH0SLn-RcDg，代碼請參見https://github.com/javieryu/nerf_bridge。

English

This work was presented at the IEEE International Conference on Robotics and Automation 2023 Workshop on Unconventional Spatial Representations. Neural radiance fields (NeRFs) are a class of implicit scene representations that model 3D environments from color images. NeRFs are expressive, and can model the complex and multi-scale geometry of real world environments, which potentially makes them a powerful tool for robotics applications. Modern NeRF training libraries can generate a photo-realistic NeRF from a static data set in just a few seconds, but are designed for offline use and require a slow pose optimization pre-computation step. In this work we propose NerfBridge, an open-source bridge between the Robot Operating System (ROS) and the popular Nerfstudio library for real-time, online training of NeRFs from a stream of images. NerfBridge enables rapid development of research on applications of NeRFs in robotics by providing an extensible interface to the efficient training pipelines and model libraries provided by Nerfstudio. As an example use case we outline a hardware setup that can be used NerfBridge to train a NeRF from images captured by a camera mounted to a quadrotor in both indoor and outdoor environments. For accompanying video https://youtu.be/EH0SLn-RcDg and code https://github.com/javieryu/nerf_bridge.

NerfBridge：將實時、在線神經輻射場訓練引入機器人技術

NerfBridge: Bringing Real-time, Online Neural Radiance Field Training to Robotics

摘要

Support