自然图像自编码器能否紧凑表征fMRI体积以进行长程动力学建模？

摘要

由于四维功能磁共振成像（fMRI）信号的高维特性，对其长程时空动态进行建模始终是核心挑战。现有的基于体素的模型虽展现出优异的性能与可解释性，但受限于巨大的内存需求，仅能捕捉有限时间窗口内的信息。为此，我们提出TABLeT（二维自编码脑潜空间变换器），通过预训练的二维自然图像自编码器对fMRI体积数据进行标记化处理，将每个三维fMRI体积压缩为紧凑的连续标记集合。该方法可在有限显存条件下，利用简易的Transformer编码器实现长序列建模。在英国生物样本库（UKB）、人类连接组计划（HCP）及ADHD-200等大规模基准测试中，TABLeT在多项任务上超越现有模型，并在相同输入条件下较当前最先进的基于体素方法显著提升计算与内存效率。此外，我们开发了基于掩码标记建模的自监督预训练策略，进一步提升模型在下游任务中的表现。本研究为脑活动的大规模可解释时空建模提供了可行路径。代码已开源：https://github.com/beotborry/TABLeT。

English

Modeling long-range spatiotemporal dynamics in functional Magnetic Resonance Imaging (fMRI) remains a key challenge due to the high dimensionality of the four-dimensional signals. Prior voxel-based models, although demonstrating excellent performance and interpretation capabilities, are constrained by prohibitive memory demands and thus can only capture limited temporal windows. To address this, we propose TABLeT (Two-dimensionally Autoencoded Brain Latent Transformer), a novel approach that tokenizes fMRI volumes using a pre-trained 2D natural image autoencoder. Each 3D fMRI volume is compressed into a compact set of continuous tokens, enabling long-sequence modeling with a simple Transformer encoder with limited VRAM. Across large-scale benchmarks including the UK-Biobank (UKB), Human Connectome Project (HCP), and ADHD-200 datasets, TABLeT outperforms existing models in multiple tasks, while demonstrating substantial gains in computational and memory efficiency over the state-of-the-art voxel-based method given the same input. Furthermore, we develop a self-supervised masked token modeling approach to pre-train TABLeT, which improves the model's performance for various downstream tasks. Our findings suggest a promising approach for scalable and interpretable spatiotemporal modeling of brain activity. Our code is available at https://github.com/beotborry/TABLeT.

自然图像自编码器能否紧凑表征fMRI体积以进行长程动力学建模？

Can Natural Image Autoencoders Compactly Tokenize fMRI Volumes for Long-Range Dynamics Modeling?

摘要

Support