Dynadiff：從持續演化的功能性磁振造影中單階段解碼影像

摘要

腦到圖像解碼技術近期因生成式AI模型的進步及大型超高場功能性磁共振成像（fMRI）數據的可用性而得到推動。然而，現有方法依賴於複雜的多階段處理流程和預處理步驟，這些步驟通常會壓縮腦部記錄的時間維度，從而限制了時間解析的腦解碼器。在此，我們介紹了Dynadiff（動態神經活動擴散用於圖像重建），這是一種新的單階段擴散模型，專為從動態演變的fMRI記錄中重建圖像而設計。我們的方法提供了三個主要貢獻。首先，與現有方法相比，Dynadiff簡化了訓練過程。其次，我們的模型在時間解析的fMRI信號上表現優於最先進的模型，特別是在高層次語義圖像重建指標上，同時在壓縮時間的預處理fMRI數據上仍保持競爭力。第三，這種方法能夠精確描述圖像表徵在腦活動中的演變過程。總體而言，這項工作為時間解析的腦到圖像解碼奠定了基礎。

English

Brain-to-image decoding has been recently propelled by the progress in generative AI models and the availability of large ultra-high field functional Magnetic Resonance Imaging (fMRI). However, current approaches depend on complicated multi-stage pipelines and preprocessing steps that typically collapse the temporal dimension of brain recordings, thereby limiting time-resolved brain decoders. Here, we introduce Dynadiff (Dynamic Neural Activity Diffusion for Image Reconstruction), a new single-stage diffusion model designed for reconstructing images from dynamically evolving fMRI recordings. Our approach offers three main contributions. First, Dynadiff simplifies training as compared to existing approaches. Second, our model outperforms state-of-the-art models on time-resolved fMRI signals, especially on high-level semantic image reconstruction metrics, while remaining competitive on preprocessed fMRI data that collapse time. Third, this approach allows a precise characterization of the evolution of image representations in brain activity. Overall, this work lays the foundation for time-resolved brain-to-image decoding.

Dynadiff：從持續演化的功能性磁振造影中單階段解碼影像

Dynadiff: Single-stage Decoding of Images from Continuously Evolving fMRI

摘要

Support