MambaEVT：使用状态空间模型的基于事件流的视觉目标跟踪

MambaEVT: Event Stream based Visual Object Tracking using State Space Model

August 20, 2024

作者: Xiao Wang, Chao wang, Shiao Wang, Xixi Wang, Zhicheng Zhao, Lin Zhu, Bo Jiang

cs.AI

摘要

基于事件相机的视觉跟踪近年来越来越受到关注，这是由于其独特的成像原理和低能耗、高动态范围以及密集时间分辨率的优势。当前基于事件的跟踪算法逐渐遇到性能瓶颈，这是由于利用视觉Transformer和静态模板进行目标对象定位。本文提出了一种新颖的基于Mamba的视觉跟踪框架，采用具有线性复杂度的状态空间模型作为骨干网络。搜索区域和目标模板被输入到视觉Mamba网络中进行同时特征提取和交互。搜索区域的输出标记将被输入到跟踪头中进行目标定位。更重要的是，我们考虑在跟踪框架中引入一种动态模板更新策略，使用Memory Mamba网络。通过考虑目标模板库中样本的多样性，并对模板存储模块进行适当调整，可以集成更有效的动态模板。动态和静态模板的有效组合使得我们基于Mamba的跟踪算法能够在多个大规模数据集（包括EventVOT、VisEvent和FE240hz）上在准确性和计算成本之间取得良好平衡。源代码将发布在https://github.com/Event-AHU/MambaEVT。

English

Event camera-based visual tracking has drawn more and more attention in recent years due to the unique imaging principle and advantages of low energy consumption, high dynamic range, and dense temporal resolution. Current event-based tracking algorithms are gradually hitting their performance bottlenecks, due to the utilization of vision Transformer and the static template for target object localization. In this paper, we propose a novel Mamba-based visual tracking framework that adopts the state space model with linear complexity as a backbone network. The search regions and target template are fed into the vision Mamba network for simultaneous feature extraction and interaction. The output tokens of search regions will be fed into the tracking head for target localization. More importantly, we consider introducing a dynamic template update strategy into the tracking framework using the Memory Mamba network. By considering the diversity of samples in the target template library and making appropriate adjustments to the template memory module, a more effective dynamic template can be integrated. The effective combination of dynamic and static templates allows our Mamba-based tracking algorithm to achieve a good balance between accuracy and computational cost on multiple large-scale datasets, including EventVOT, VisEvent, and FE240hz. The source code will be released on https://github.com/Event-AHU/MambaEVT