Dynadiff：从持续演化的fMRI中单阶段解码图像

摘要

脑到图像解码技术近期得益于生成式AI模型的进步以及大规模超高场功能磁共振成像（fMRI）数据的可用性而取得显著进展。然而，现有方法依赖于复杂的多阶段处理流程和预处理步骤，这些步骤通常压缩了脑记录的时间维度，从而限制了时间分辨型脑解码器的性能。在此，我们提出了Dynadiff（动态神经活动扩散图像重建模型），这是一种专为从动态演变的fMRI记录中重建图像而设计的新型单阶段扩散模型。我们的方法具有三大贡献：首先，与现有方法相比，Dynadiff简化了训练过程；其次，该模型在处理时间分辨的fMRI信号时，尤其是在高级语义图像重建指标上，超越了当前最先进的模型，同时在处理已压缩时间维度的预处理fMRI数据时也保持竞争力；最后，此方法能够精确刻画图像表征在大脑活动中的演变过程。总体而言，本研究为时间分辨型脑到图像解码奠定了基石。

English

Brain-to-image decoding has been recently propelled by the progress in generative AI models and the availability of large ultra-high field functional Magnetic Resonance Imaging (fMRI). However, current approaches depend on complicated multi-stage pipelines and preprocessing steps that typically collapse the temporal dimension of brain recordings, thereby limiting time-resolved brain decoders. Here, we introduce Dynadiff (Dynamic Neural Activity Diffusion for Image Reconstruction), a new single-stage diffusion model designed for reconstructing images from dynamically evolving fMRI recordings. Our approach offers three main contributions. First, Dynadiff simplifies training as compared to existing approaches. Second, our model outperforms state-of-the-art models on time-resolved fMRI signals, especially on high-level semantic image reconstruction metrics, while remaining competitive on preprocessed fMRI data that collapse time. Third, this approach allows a precise characterization of the evolution of image representations in brain activity. Overall, this work lays the foundation for time-resolved brain-to-image decoding.

Dynadiff：从持续演化的fMRI中单阶段解码图像

Dynadiff: Single-stage Decoding of Images from Continuously Evolving fMRI

摘要

Support