MADrive：记忆增强型驾驶场景建模

摘要

近期场景重建技术的进步推动了利用3D高斯溅射实现自动驾驶（AD）环境的高度真实建模。然而，这些重建结果仍紧密依赖于原始观测数据，难以支持对显著改变或全新驾驶场景的逼真合成。本研究提出了MADrive，一种记忆增强型重建框架，旨在通过从大规模外部记忆库中检索视觉相似的3D资产来替换观测到的车辆，从而扩展现有场景重建方法的能力。具体而言，我们发布了MAD-Cars，一个包含约70K段360度野外拍摄的汽车视频的精选数据集，并介绍了一个检索模块，该模块能在记忆库中找到最相似的汽车实例，从视频中重建相应的3D资产，并通过方向对齐和重光照技术将其整合到目标场景中。替换后的车辆在场景中提供了完整的多视角表示，使得大幅改变的配置也能实现逼真合成，如我们的实验所展示。项目页面：https://yandex-research.github.io/madrive/

English

Recent advances in scene reconstruction have pushed toward highly realistic modeling of autonomous driving (AD) environments using 3D Gaussian splatting. However, the resulting reconstructions remain closely tied to the original observations and struggle to support photorealistic synthesis of significantly altered or novel driving scenarios. This work introduces MADrive, a memory-augmented reconstruction framework designed to extend the capabilities of existing scene reconstruction methods by replacing observed vehicles with visually similar 3D assets retrieved from a large-scale external memory bank. Specifically, we release MAD-Cars, a curated dataset of {sim}70K 360{\deg} car videos captured in the wild and present a retrieval module that finds the most similar car instances in the memory bank, reconstructs the corresponding 3D assets from video, and integrates them into the target scene through orientation alignment and relighting. The resulting replacements provide complete multi-view representations of vehicles in the scene, enabling photorealistic synthesis of substantially altered configurations, as demonstrated in our experiments. Project page: https://yandex-research.github.io/madrive/

MADrive：记忆增强型驾驶场景建模

MADrive: Memory-Augmented Driving Scene Modeling

摘要

Support