SimScale：基于大规模真实世界模拟的驾驶学习

摘要

实现完全自动驾驶系统需在广泛场景中学习理性决策，包括安全关键场景和分布外场景。然而此类案例在人类专家收集的真实世界数据集中占比不足。为弥补数据多样性缺失，我们提出一种新颖的可扩展仿真框架，能够在现有驾驶日志基础上合成海量未见状态。该流程采用先进神经渲染技术结合响应式环境，通过扰动自车轨迹生成高保真多视角观测数据。此外，我们为这些新模拟状态开发了伪专家轨迹生成机制以提供动作监督。基于合成数据，我们发现对真实样本与模拟样本进行简单协同训练，可使多种规划方法在挑战性真实基准测试中的鲁棒性和泛化性显著提升——在navhard上最高提升6.8 EPDMS，在navtest上提升2.9。更重要的是，仅通过增加模拟数据（无需额外真实数据流），这种策略改进就能实现平滑扩展。我们进一步揭示了此类虚实融合学习系统（命名为SimScale）的关键发现，包括伪专家设计原则及不同策略架构的扩展特性。我们的仿真数据与代码将开源发布。

English

Achieving fully autonomous driving systems requires learning rational decisions in a wide span of scenarios, including safety-critical and out-of-distribution ones. However, such cases are underrepresented in real-world corpus collected by human experts. To complement for the lack of data diversity, we introduce a novel and scalable simulation framework capable of synthesizing massive unseen states upon existing driving logs. Our pipeline utilizes advanced neural rendering with a reactive environment to generate high-fidelity multi-view observations controlled by the perturbed ego trajectory. Furthermore, we develop a pseudo-expert trajectory generation mechanism for these newly simulated states to provide action supervision. Upon the synthesized data, we find that a simple co-training strategy on both real-world and simulated samples can lead to significant improvements in both robustness and generalization for various planning methods on challenging real-world benchmarks, up to +6.8 EPDMS on navhard and +2.9 on navtest. More importantly, such policy improvement scales smoothly by increasing simulation data only, even without extra real-world data streaming in. We further reveal several crucial findings of such a sim-real learning system, which we term SimScale, including the design of pseudo-experts and the scaling properties for different policy architectures. Our simulation data and code would be released.

SimScale：基于大规模真实世界模拟的驾驶学习

SimScale: Learning to Drive via Real-World Simulation at Scale

摘要

Support