无处不在的全面跟踪

摘要

我们提出了一种新的测试时间优化方法，用于从视频序列中估计密集且长距离的运动。先前的光流或粒子视频跟踪算法通常在有限的时间窗口内运行，难以跟踪遮挡部分并保持估计运动轨迹的全局一致性。我们提出了一种完整且全局一致的运动表示，称为全景运动（OmniMotion），允许准确地对视频中每个像素的全长运动进行估计。全景运动使用准三维规范体表示视频，并通过局部空间和规范空间之间的双射执行逐像素跟踪。这种表示使我们能够确保全局一致性，跟踪遮挡部分，并对相机和物体运动的任何组合进行建模。在TAP-Vid基准和真实世界素材上进行的广泛评估显示，我们的方法在定量和定性上均大幅优于先前的最新方法。有关更多结果，请访问我们的项目页面：http://omnimotion.github.io/

English

We present a new test-time optimization method for estimating dense and long-range motion from a video sequence. Prior optical flow or particle video tracking algorithms typically operate within limited temporal windows, struggling to track through occlusions and maintain global consistency of estimated motion trajectories. We propose a complete and globally consistent motion representation, dubbed OmniMotion, that allows for accurate, full-length motion estimation of every pixel in a video. OmniMotion represents a video using a quasi-3D canonical volume and performs pixel-wise tracking via bijections between local and canonical space. This representation allows us to ensure global consistency, track through occlusions, and model any combination of camera and object motion. Extensive evaluations on the TAP-Vid benchmark and real-world footage show that our approach outperforms prior state-of-the-art methods by a large margin both quantitatively and qualitatively. See our project page for more results: http://omnimotion.github.io/

无处不在的全面跟踪

Tracking Everything Everywhere All at Once

摘要

Support