GenieDrive：迈向基于4D占据导引视频生成的物理感知驾驶世界模型

GenieDrive: Towards Physics-Aware Driving World Model with 4D Occupancy Guided Video Generation

December 14, 2025

作者: Zhenya Yang, Zhe Liu, Yuxiang Lu, Liping Hou, Chenxuan Miao, Siyi Peng, Bailan Feng, Xiang Bai, Hengshuang Zhao

cs.AI

摘要

具备物理感知的驾驶世界模型对于行驶规划、分布外数据合成和闭环评估至关重要。然而，现有方法通常依赖单一扩散模型直接将驾驶动作映射为视频，这导致学习困难并产生物理不一致的输出。为克服这些挑战，我们提出GenieDrive——一个专为物理感知驾驶视频生成设计的新型框架。该方法首先生成包含物理信息的4D占据场，作为后续视频生成的物理基础。4D占据场蕴含丰富的物理信息，包括高分辨率三维结构与动态特性。为有效压缩此类高分辨率占据场，我们提出一种变分自编码器，将占据场编码为潜在三平面表示，使潜在空间尺寸降至原有方法的58%。我们进一步引入互控注意力机制来精确建模控制信号对占据场演化的影响，并以端到端方式联合训练变分自编码器与预测模块以最大化预测精度。这些设计共同实现了推理速度41帧/秒下预测mIoU指标提升7.2%，且仅需347万参数。此外，视频生成模型中引入的归一化多视角注意力机制，能够基于4D占据场指导生成多视角驾驶视频，使FVD指标显著降低20.7%。实验表明，GenieDrive能够实现高度可控、多视角一致且具备物理感知的驾驶视频生成。

English

Physics-aware driving world model is essential for drive planning, out-of-distribution data synthesis, and closed-loop evaluation. However, existing methods often rely on a single diffusion model to directly map driving actions to videos, which makes learning difficult and leads to physically inconsistent outputs. To overcome these challenges, we propose GenieDrive, a novel framework designed for physics-aware driving video generation. Our approach starts by generating 4D occupancy, which serves as a physics-informed foundation for subsequent video generation. 4D occupancy contains rich physical information, including high-resolution 3D structures and dynamics. To facilitate effective compression of such high-resolution occupancy, we propose a VAE that encodes occupancy into a latent tri-plane representation, reducing the latent size to only 58% of that used in previous methods. We further introduce Mutual Control Attention (MCA) to accurately model the influence of control on occupancy evolution, and we jointly train the VAE and the subsequent prediction module in an end-to-end manner to maximize forecasting accuracy. Together, these designs yield a 7.2% improvement in forecasting mIoU at an inference speed of 41 FPS, while using only 3.47 M parameters. Additionally, a Normalized Multi-View Attention is introduced in the video generation model to generate multi-view driving videos with guidance from our 4D occupancy, significantly improving video quality with a 20.7% reduction in FVD. Experiments demonstrate that GenieDrive enables highly controllable, multi-view consistent, and physics-aware driving video generation.