無處不在的全面追蹤

摘要

我們提出了一種新的測試時間優化方法，用於從視頻序列中估計密集且長距離運動。先前的光流或粒子視頻跟踪算法通常在有限的時間窗口內運作，難以跟踪遮蔽物並保持估計運動軌跡的全局一致性。我們提出了一種完整且全局一致的運動表示，稱為OmniMotion，它允許對視頻中的每個像素進行準確的全長度運動估計。OmniMotion使用幾乎三維的標準體表示視頻，並通過局部空間和標準空間之間的雙射進行像素級跟踪。這種表示使我們能夠確保全局一致性，穿越遮蔽物，並模擬任何相機和物體運動的組合。在TAP-Vid基準測試和現實世界影片上進行了廣泛評估，結果顯示我們的方法在定量和定性上均大幅優於先前的最先進方法。請查看我們的項目頁面以獲取更多結果：http://omnimotion.github.io/

English

We present a new test-time optimization method for estimating dense and long-range motion from a video sequence. Prior optical flow or particle video tracking algorithms typically operate within limited temporal windows, struggling to track through occlusions and maintain global consistency of estimated motion trajectories. We propose a complete and globally consistent motion representation, dubbed OmniMotion, that allows for accurate, full-length motion estimation of every pixel in a video. OmniMotion represents a video using a quasi-3D canonical volume and performs pixel-wise tracking via bijections between local and canonical space. This representation allows us to ensure global consistency, track through occlusions, and model any combination of camera and object motion. Extensive evaluations on the TAP-Vid benchmark and real-world footage show that our approach outperforms prior state-of-the-art methods by a large margin both quantitatively and qualitatively. See our project page for more results: http://omnimotion.github.io/

無處不在的全面追蹤

Tracking Everything Everywhere All at Once

摘要

Support