元学习上下文学习实现免训练的跨被试脑信号解码

Meta-learning In-Context Enables Training-Free Cross Subject Brain Decoding

April 9, 2026

作者: Mu Nan, Muquan Yu, Weijian Mai, Jacob S. Prince, Hossein Adeli, Rui Zhang, Jiahang Cao, Benjamin Becker, John A. Pyles, Margaret M. Henderson, Chunfeng Song, Nikolaus Kriegeskorte, Michael J. Tarr, Xiaoqing Hu, Andrew F. Luo

cs.AI

摘要

脑信号视觉解码是计算机视觉与神经科学交叉领域的关键挑战，需要构建连接神经表征与视觉计算模型的方法。该领域的共同目标是实现可泛化的跨被试模型，而主要障碍在于个体间神经表征存在显著差异，目前仍需为每位被试训练定制模型或进行单独微调。为解决这一难题，我们提出一种基于fMRI的语义视觉解码元优化方法，无需微调即可泛化至新被试。仅需通过新被试少量图像-脑激活示例进行条件化，我们的模型便能快速推断其独特的神经编码模式，从而实现稳健高效的视觉解码。该方法专门针对新被试编码模型的上下文学习进行优化，并通过分层推理执行编码器逆变换的解码过程：首先在多脑区构建多刺激-响应上下文，估计单体素的视觉响应编码器参数；随后在多个体素上构建编码器参数与响应值的上下文，执行聚合式功能逆变换。实验表明，该方法在不同视觉骨干网络上均实现了强大的跨被试、跨扫描仪泛化能力，且无需重新训练或微调。此外，我们的方法既不需要解剖结构对齐，也不依赖刺激重叠。这项研究为构建非侵入式脑解码的通用基础模型迈出了关键一步。

English

Visual decoding from brain signals is a key challenge at the intersection of computer vision and neuroscience, requiring methods that bridge neural representations and computational models of vision. A field-wide goal is to achieve generalizable, cross-subject models. A major obstacle towards this goal is the substantial variability in neural representations across individuals, which has so far required training bespoke models or fine-tuning separately for each subject. To address this challenge, we introduce a meta-optimized approach for semantic visual decoding from fMRI that generalizes to novel subjects without any fine-tuning. By simply conditioning on a small set of image-brain activation examples from the new individual, our model rapidly infers their unique neural encoding patterns to facilitate robust and efficient visual decoding. Our approach is explicitly optimized for in-context learning of the new subject's encoding model and performs decoding by hierarchical inference, inverting the encoder. First, for multiple brain regions, we estimate the per-voxel visual response encoder parameters by constructing a context over multiple stimuli and responses. Second, we construct a context consisting of encoder parameters and response values over multiple voxels to perform aggregated functional inversion. We demonstrate strong cross-subject and cross-scanner generalization across diverse visual backbones without retraining or fine-tuning. Moreover, our approach requires neither anatomical alignment nor stimulus overlap. This work is a critical step towards a generalizable foundation model for non-invasive brain decoding.