基于时空注意力链的快速四维网格生成

摘要

4D网格生成近期已成为从视频中恢复动态三维结构的一种强大范式，但现有方法仍存在速度慢、计算成本高、难以扩展到更长序列等问题。我们提出了一种免训练方案，能加速4D网格生成并提升时间对应质量。关键发现是：4D骨架中的时间对应关系在其生成的网格视觉上变得精确之前很早就已显现。我们利用这一现象，设计了一个名为“时空注意力链”的通用框架，可在空间与时间维度传播信息。该链以锚定网格上的顶点为起点，将顶点映射为潜在标记，随后沿潜在空间中的时间对应关系进行追踪，并通过潜在到顶点的注意力机制恢复各帧的特定顶点。这一设计避免了昂贵的显式匹配，同时保留了锚定网格细节，从而改善了动态网格几何结构与时间一致性。与现有最优方法相比，我们的方法能在9秒内生成一个4D网格，实现13倍加速，同时生成更高质量的结果。此外，我们的方法可扩展至长达16倍的视频序列而不降低网格质量。在生成任务之外，改进的对应关系使方法在两个下游任务（二维目标跟踪与四维跟踪）上具备竞争力强的零样本性能。我们还展示了本框架能够实现可靠的相机估计，而这一能力是先前4D网格生成方法所不具备的。

English

4D mesh generation has recently emerged as a powerful paradigm for recovering dynamic 3D structure from videos, but existing methods remain slow, computationally expensive, and difficult to scale to longer sequences. We introduce a training-free approach that accelerates 4D mesh generation while improving temporal correspondence quality. Our key observation is that temporal correspondences emerge inside a 4D backbone long before its generated meshes become visually accurate. We exploit this with a general framework we call Spatio-Temporal Attention Chain which propagates information across space and time. Starting from vertices on an anchor mesh, the chain maps vertices to latent tokens. It then follows temporal correspondences in latent space, and recovers frame-specific vertices through latent-to-vertex attention. This design avoids expensive explicit matching while preserving anchor mesh details and thereby improving dynamic mesh geometry and temporal consistency. Compared to state-of-the-art, our method generates a 4D mesh in 9 seconds, achieving a 13times speedup while producing higher-quality results. Moreover, our approach scales to videos up to 16times longer without degrading mesh quality. Beyond generation, the improved correspondences enable competitive zero-shot performance on two downstream tasks: 2D object tracking and 4D tracking. We further show that our framework enables reliable camera estimation, a capability not supported by prior 4D mesh generation methods.