MADrive：記憶增強型駕駛場景建模

摘要

近期場景重建技術的進步，已推動了利用3D高斯潑濺技術對自動駕駛（AD）環境進行高度逼真建模的發展。然而，這些重建結果仍緊密依賴於原始觀測數據，難以支持對顯著改變或全新駕駛場景的逼真合成。本研究提出了MADrive，這是一個記憶增強的重建框架，旨在通過從大規模外部記憶庫中檢索視覺相似的3D資產來替換觀測到的車輛，從而擴展現有場景重建方法的能力。具體而言，我們發布了MAD-Cars，這是一個精心策劃的數據集，包含約70K個在野外捕捉的360度汽車視頻，並提出了一個檢索模塊，該模塊在記憶庫中尋找最相似的汽車實例，從視頻中重建相應的3D資產，並通過方向對齊和重新照明將其整合到目標場景中。由此產生的替換提供了場景中車輛的完整多視圖表示，使得在實驗中展示的顯著改變配置的逼真合成成為可能。項目頁面：https://yandex-research.github.io/madrive/

English

Recent advances in scene reconstruction have pushed toward highly realistic modeling of autonomous driving (AD) environments using 3D Gaussian splatting. However, the resulting reconstructions remain closely tied to the original observations and struggle to support photorealistic synthesis of significantly altered or novel driving scenarios. This work introduces MADrive, a memory-augmented reconstruction framework designed to extend the capabilities of existing scene reconstruction methods by replacing observed vehicles with visually similar 3D assets retrieved from a large-scale external memory bank. Specifically, we release MAD-Cars, a curated dataset of {sim}70K 360{\deg} car videos captured in the wild and present a retrieval module that finds the most similar car instances in the memory bank, reconstructs the corresponding 3D assets from video, and integrates them into the target scene through orientation alignment and relighting. The resulting replacements provide complete multi-view representations of vehicles in the scene, enabling photorealistic synthesis of substantially altered configurations, as demonstrated in our experiments. Project page: https://yandex-research.github.io/madrive/

MADrive：記憶增強型駕駛場景建模

MADrive: Memory-Augmented Driving Scene Modeling

摘要

Support