透过大脑看世界：从人类脑信号重建视觉知觉图像

摘要

眼见为实，然而，人类视觉知觉与认知相互交织的基本机制仍然是一个谜。由于近年来神经科学和人工智能的快速发展，我们已经能够记录视觉诱发的大脑活动，并通过计算方法模拟视觉知觉能力。本文关注通过基于便携式脑信号（即脑电图（EEG）数据）重建观察到的图像来重建视觉刺激。由于EEG信号以时间序列格式动态变化且因其嘈杂而臭名昭著，处理和提取有用信息需要更多专门的努力；在本文中，我们提出了一个名为NeuroImagen的全面流程，用于从EEG信号中重建视觉刺激图像。具体而言，我们结合了一种新颖的多层次感知信息解码，以从给定的EEG数据中得出多粒度的输出。然后，一个潜在扩散模型将利用提取的信息重建高分辨率的视觉刺激图像。实验结果已经证明了图像重建的有效性以及我们提出的方法在定量性能上的优越表现。

English

Seeing is believing, however, the underlying mechanism of how human visual perceptions are intertwined with our cognitions is still a mystery. Thanks to the recent advances in both neuroscience and artificial intelligence, we have been able to record the visually evoked brain activities and mimic the visual perception ability through computational approaches. In this paper, we pay attention to visual stimuli reconstruction by reconstructing the observed images based on portably accessible brain signals, i.e., electroencephalography (EEG) data. Since EEG signals are dynamic in the time-series format and are notorious to be noisy, processing and extracting useful information requires more dedicated efforts; In this paper, we propose a comprehensive pipeline, named NeuroImagen, for reconstructing visual stimuli images from EEG signals. Specifically, we incorporate a novel multi-level perceptual information decoding to draw multi-grained outputs from the given EEG data. A latent diffusion model will then leverage the extracted information to reconstruct the high-resolution visual stimuli images. The experimental results have illustrated the effectiveness of image reconstruction and superior quantitative performance of our proposed method.

透过大脑看世界：从人类脑信号重建视觉知觉图像

Seeing through the Brain: Image Reconstruction of Visual Perception from Human Brain Signals

摘要

Support