Valdi：价值扩散世界模型

摘要

世界模型能够实现模型预测控制（MPC），但这需要动态预测既足够快速以支持在线使用，又足够灵活以表征不确定的未来。扩散模型为建模不确定动态提供了天然机制，但其迭代推理过程使其难以用于低延迟的潜在规划。我们通过价值扩散世界模型（Valdi）弥合了这一差距，将端到端在线训练用于MPC与潜在扩散动态模型相结合。在CarRacing环境的初步实验中，我们展示了Valdi在训练和推理时仅使用单步扩散，便能与确定性MLP基线表现相当。实验揭示了该框架中预测多模态性与控制性能之间的权衡。代码见https://github.com/Kit115/ValueDiffusionWorldModels。

English

World models can enable Model Predictive Control (MPC), but this requires dynamics prediction that is both fast enough for online use and expressive enough to represent uncertain futures. Diffusion models offer a natural mechanism for modeling uncertain dynamics, yet their iterative inference procedure makes them difficult to use for low-latency latent planning. We bridge this gap with Value Diffusion World Models (Valdi), combining end-to-end online training for MPC with a latent diffusion dynamics model. In preliminary experiments on the CarRacing environment, we show that Valdi, using a single diffusion step at both training and inference, matches a deterministic MLP baseline. Our experiments expose a trade-off between predictive multimodality and control performance in this setup. Code is available at https://github.com/Kit115/ValueDiffusionWorldModels.