RoCo-Sim：通过前景模拟提升路侧协同感知能力

RoCo-Sim: Enhancing Roadside Collaborative Perception through Foreground Simulation

March 13, 2025

作者: Yuwen Du, Anning Hu, Zichen Chao, Yifan Lu, Junhao Ge, Genjia Liu, Weitao Wu, Lanjun Wang, Siheng Chen

cs.AI

摘要

路侧协同感知系统是指多个路侧单元协作共享其感知数据，以辅助车辆提升环境感知能力。现有路侧感知方法主要关注模型设计，却忽视了校准误差、信息稀疏和多视角一致性等数据问题，导致在最新发布的数据集上表现不佳。为显著提升路侧协同感知并解决关键数据问题，我们提出了首个路侧协同感知仿真框架RoCo-Sim。RoCo-Sim能够通过动态前景编辑和单图像全场景风格迁移，生成多样化且多视角一致的路侧仿真数据。RoCo-Sim包含四个核心组件：(1) 相机外参优化，确保路侧摄像头精确的3D到2D投影；(2) 创新的多视角遮挡感知采样器(MOAS)，决定多样数字资产在3D空间中的布局；(3) DepthSAM，从单帧固定视角图像中创新建模前景与背景关系，保证前景的多视角一致性；(4) 可扩展的后处理工具包，通过风格迁移等增强手段生成更真实丰富的场景。RoCo-Sim显著提升了路侧3D物体检测性能，在Rcooper-Intersection和TUMTraf-V2X数据集上的AP70指标分别超越当前最优方法83.74和83.12。RoCo-Sim填补了路侧感知仿真领域的关键空白。代码与预训练模型即将发布：https://github.com/duyuwen-duen/RoCo-Sim。

English

Roadside Collaborative Perception refers to a system where multiple roadside units collaborate to pool their perceptual data, assisting vehicles in enhancing their environmental awareness. Existing roadside perception methods concentrate on model design but overlook data issues like calibration errors, sparse information, and multi-view consistency, leading to poor performance on recent published datasets. To significantly enhance roadside collaborative perception and address critical data issues, we present the first simulation framework RoCo-Sim for road-side collaborative perception. RoCo-Sim is capable of generating diverse, multi-view consistent simulated roadside data through dynamic foreground editing and full-scene style transfer of a single image. RoCo-Sim consists of four components: (1) Camera Extrinsic Optimization ensures accurate 3D to 2D projection for roadside cameras; (2) A novel Multi-View Occlusion-Aware Sampler (MOAS) determines the placement of diverse digital assets within 3D space; (3) DepthSAM innovatively models foreground-background relationships from single-frame fixed-view images, ensuring multi-view consistency of foreground; and (4) Scalable Post-Processing Toolkit generates more realistic and enriched scenes through style transfer and other enhancements. RoCo-Sim significantly improves roadside 3D object detection, outperforming SOTA methods by 83.74 on Rcooper-Intersection and 83.12 on TUMTraf-V2X for AP70. RoCo-Sim fills a critical gap in roadside perception simulation. Code and pre-trained models will be released soon: https://github.com/duyuwen-duen/RoCo-Sim