EvTexture：用於視頻超分辨率的事件驅動紋理增強

EvTexture: Event-driven Texture Enhancement for Video Super-Resolution

June 19, 2024

作者: Dachun Kai, Jiayao Lu, Yueyi Zhang, Xiaoyan Sun

cs.AI

摘要

基於事件的視覺因具有高時間解析度和高動態範圍等獨特特性，近來在視頻超分辨率（VSR）中被廣泛關注，用於增強流估計和時間對齊。本文提出了一種新的VSR方法，名為EvTexture，該方法利用事件信號進行紋理增強，而非用於運動學習。EvTexture利用事件的高頻細節更好地恢復VSR中的紋理區域。在EvTexture中，引入了一個新的紋理增強分支。我們進一步引入了一個迭代紋理增強模塊，逐步探索高時間解析度的事件信息以進行紋理修復。這使得在多次迭代中逐步改進紋理區域，從而獲得更準確和豐富的高分辨率細節。實驗結果表明，我們的EvTexture在四個數據集上實現了最先進的性能。對於具有豐富紋理的Vid4數據集，我們的方法與最近的基於事件的方法相比，可以獲得高達4.67dB的增益。代碼：https://github.com/DachunKai/EvTexture。

English

Event-based vision has drawn increasing attention due to its unique characteristics, such as high temporal resolution and high dynamic range. It has been used in video super-resolution (VSR) recently to enhance the flow estimation and temporal alignment. Rather than for motion learning, we propose in this paper the first VSR method that utilizes event signals for texture enhancement. Our method, called EvTexture, leverages high-frequency details of events to better recover texture regions in VSR. In our EvTexture, a new texture enhancement branch is presented. We further introduce an iterative texture enhancement module to progressively explore the high-temporal-resolution event information for texture restoration. This allows for gradual refinement of texture regions across multiple iterations, leading to more accurate and rich high-resolution details. Experimental results show that our EvTexture achieves state-of-the-art performance on four datasets. For the Vid4 dataset with rich textures, our method can get up to 4.67dB gain compared with recent event-based methods. Code: https://github.com/DachunKai/EvTexture.