RoCo-Sim：通過前景模擬提升路側協同感知能力

RoCo-Sim: Enhancing Roadside Collaborative Perception through Foreground Simulation

March 13, 2025

作者: Yuwen Du, Anning Hu, Zichen Chao, Yifan Lu, Junhao Ge, Genjia Liu, Weitao Wu, Lanjun Wang, Siheng Chen

cs.AI

摘要

路側協同感知系統，意指多個路側單元協作匯聚其感知數據，以協助車輛提升環境感知能力。現有的路側感知方法專注於模型設計，卻忽視了校準誤差、信息稀疏性及多視角一致性等數據問題，導致在近期發布的數據集上表現欠佳。為顯著提升路側協同感知並解決關鍵數據問題，我們提出了首個路側協同感知模擬框架RoCo-Sim。RoCo-Sim能夠通過單一圖像的動態前景編輯與全場景風格遷移，生成多樣化且多視角一致的模擬路側數據。RoCo-Sim包含四個組件：(1) 相機外參優化，確保路側相機的3D到2D投影精確；(2) 新穎的多視角遮擋感知採樣器(MOAS)，決定多樣化數字資產在3D空間中的佈局；(3) DepthSAM創新性地從單幀固定視角圖像中建模前景與背景關係，確保前景的多視角一致性；(4) 可擴展的後處理工具包，通過風格遷移及其他增強手段生成更為真實且豐富的場景。RoCo-Sim顯著提升了路側3D物體檢測性能，在Rcooper-Intersection和TUMTraf-V2X數據集上，AP70指標分別超越現有最佳方法83.74和83.12。RoCo-Sim填補了路側感知模擬領域的關鍵空白。代碼及預訓練模型即將發布：https://github.com/duyuwen-duen/RoCo-Sim

English

Roadside Collaborative Perception refers to a system where multiple roadside units collaborate to pool their perceptual data, assisting vehicles in enhancing their environmental awareness. Existing roadside perception methods concentrate on model design but overlook data issues like calibration errors, sparse information, and multi-view consistency, leading to poor performance on recent published datasets. To significantly enhance roadside collaborative perception and address critical data issues, we present the first simulation framework RoCo-Sim for road-side collaborative perception. RoCo-Sim is capable of generating diverse, multi-view consistent simulated roadside data through dynamic foreground editing and full-scene style transfer of a single image. RoCo-Sim consists of four components: (1) Camera Extrinsic Optimization ensures accurate 3D to 2D projection for roadside cameras; (2) A novel Multi-View Occlusion-Aware Sampler (MOAS) determines the placement of diverse digital assets within 3D space; (3) DepthSAM innovatively models foreground-background relationships from single-frame fixed-view images, ensuring multi-view consistency of foreground; and (4) Scalable Post-Processing Toolkit generates more realistic and enriched scenes through style transfer and other enhancements. RoCo-Sim significantly improves roadside 3D object detection, outperforming SOTA methods by 83.74 on Rcooper-Intersection and 83.12 on TUMTraf-V2X for AP70. RoCo-Sim fills a critical gap in roadside perception simulation. Code and pre-trained models will be released soon: https://github.com/duyuwen-duen/RoCo-Sim