ROOM：一個基於物理的連續體機器人模擬器，用於生成逼真的醫療數據集

ROOM: A Physics-Based Continuum Robot Simulator for Photorealistic Medical Datasets Generation

September 16, 2025

作者: Salvatore Esposito, Matías Mattamala, Daniel Rebain, Francis Xiatian Zhang, Kevin Dhaliwal, Mohsen Khadem, Subramanian Ramamoorthy

cs.AI

摘要

連續體機器人正在革新支氣管鏡檢查程序，能夠深入複雜的肺部氣道並實現精準介入治療。然而，其發展受限於缺乏逼真的訓練與測試環境：由於倫理限制和患者安全考慮，真實數據難以收集，而開發自主算法又需要真實的影像和物理反饋。我們推出了ROOM（醫學中的真實光學觀測），這是一個專為生成逼真支氣管鏡訓練數據而設計的綜合模擬框架。通過利用患者的CT掃描，我們的流程渲染出多模態傳感器數據，包括帶有真實噪聲和光澤反射的RGB圖像、度量深度圖、表面法線、光流以及醫學相關尺度的點雲。我們在兩個醫療機器人經典任務——多視角姿態估計和單目深度估計中驗證了ROOM生成的數據，展示了頂尖方法在轉移至這些醫療場景時必須克服的多樣挑戰。此外，我們證明ROOM產生的數據可用於微調現有的深度估計模型，以克服這些挑戰，同時也支持如導航等其他下游應用。我們期待ROOM能夠實現大規模數據生成，涵蓋臨床環境中難以捕捉的多樣患者解剖結構和手術場景。代碼與數據：https://github.com/iamsalvatore/room。

English

Continuum robots are advancing bronchoscopy procedures by accessing complex lung airways and enabling targeted interventions. However, their development is limited by the lack of realistic training and test environments: Real data is difficult to collect due to ethical constraints and patient safety concerns, and developing autonomy algorithms requires realistic imaging and physical feedback. We present ROOM (Realistic Optical Observation in Medicine), a comprehensive simulation framework designed for generating photorealistic bronchoscopy training data. By leveraging patient CT scans, our pipeline renders multi-modal sensor data including RGB images with realistic noise and light specularities, metric depth maps, surface normals, optical flow and point clouds at medically relevant scales. We validate the data generated by ROOM in two canonical tasks for medical robotics -- multi-view pose estimation and monocular depth estimation, demonstrating diverse challenges that state-of-the-art methods must overcome to transfer to these medical settings. Furthermore, we show that the data produced by ROOM can be used to fine-tune existing depth estimation models to overcome these challenges, also enabling other downstream applications such as navigation. We expect that ROOM will enable large-scale data generation across diverse patient anatomies and procedural scenarios that are challenging to capture in clinical settings. Code and data: https://github.com/iamsalvatore/room.