Stream4D：面向流式自回归扩散视频模型的4D一致性

摘要

流式自回归扩散模型能够实现实时、长时域视频生成，但其训练目标优化的是局部帧预测，而非连贯世界的几何与动态：长时间生成会累积几何漂移，退化为静态或不自然的运动。近期双向方法利用基于3D高斯泼溅重建的奖励信号来解决该问题。然而，单一的刚性3D重建无法建模动态场景，因此此类评判器会将真实的物体运动视为重建误差加以惩罚，并通过冻结视频来获得最大奖励。这种捷径在自回归设置中尤为有害，因为每个块都可能传播已经静态化的配置。在本工作中，我们提出Stream4D，用显式建模场景动态的前馈4D重建奖励取代静态评判器，使连贯的运动能够获得高一致性奖励。为进一步引导运动的幅度与质量，我们引入运动先验，奖励自然的场景流幅度，同时惩罚抖动与非刚性伪影。我们的最终方案将这两项与轻量级感知锚点相结合。在多种自回归视频主干网络和多种生成时域上，Stream4D均能提升4D重建质量，更有效地保持运动，并获得更高的人类对齐偏好。项目主页：https://banyuanhao.github.io/Stream4D/

English

Streaming autoregressive diffusion models enable real-time, long-horizon video generation, but their training objectives optimize local frame prediction rather than the geometry and dynamics of a coherent world: long rollouts accumulate geometric drift and degrade into static or unnatural motion. Recent bidirectional approaches address this problem using rewards signals built upon 3D Gaussian-Splatting reconstruction. However, a single rigid 3d reconstruction cannot model a dynamic scene, so this critic penalizes genuine object motion as reconstruction error and is maximized by freezing the video. This shortcut is especially detrimental in the AR setting, where each chunk can propagate an already-static configuration. In this work, we propose Stream4D, which replaces the static critic with a feed-forward 4D reconstruction reward that explicitly models scene dynamics, allowing coherent motion to receive high consistency rewards. To further guide motion magnitude and quality, we add a motion prior that rewards natural scene-flow magnitude while penalizing jitter and non-rigid artifacts. Our final recipe combines these two terms with a lightweight perceptual anchor. Across various autoregressive video backbones and various generation horizons, Stream4D improves 4D reconstruction quality, preserves motion more effectively, and achieves higher human-aligned preference. Project page: https://banyuanhao.github.io/Stream4D/