Kalman啟發的特徵傳播用於視頻人臉超分辨率

Kalman-Inspired Feature Propagation for Video Face Super-Resolution

August 9, 2024

作者: Ruicheng Feng, Chongyi Li, Chen Change Loy

cs.AI

摘要

儘管臉部影像超分辨率取得了令人期待的進展，但視頻臉部超分辨率仍相對未被充分探索。現有方法要麼將一般視頻超分辨率網絡適應於臉部數據集，要麼將已建立的臉部影像超分辨率模型獨立應用於個別視頻幀上。這些範式在重建面部細節或保持時間一致性方面均面臨挑戰。為解決這些問題，我們引入了一個名為Kalman靈感特徵傳播（KEEP）的新框架，旨在隨時間保持穩定的臉部先驗。Kalman濾波原則為我們的方法提供了一種遞歸能力，利用先前恢復的幀的信息來引導和調節當前幀的恢復過程。大量實驗證明了我們的方法在一致捕捉視頻幀間面部細節方面的有效性。代碼和視頻演示可在https://jnjaby.github.io/projects/KEEP找到。

English

Despite the promising progress of face image super-resolution, video face super-resolution remains relatively under-explored. Existing approaches either adapt general video super-resolution networks to face datasets or apply established face image super-resolution models independently on individual video frames. These paradigms encounter challenges either in reconstructing facial details or maintaining temporal consistency. To address these issues, we introduce a novel framework called Kalman-inspired Feature Propagation (KEEP), designed to maintain a stable face prior over time. The Kalman filtering principles offer our method a recurrent ability to use the information from previously restored frames to guide and regulate the restoration process of the current frame. Extensive experiments demonstrate the effectiveness of our method in capturing facial details consistently across video frames. Code and video demo are available at https://jnjaby.github.io/projects/KEEP.