Neuro2Semantic：一种用于人类颅内脑电图连续语言语义重建的迁移学习框架

Neuro2Semantic: A Transfer Learning Framework for Semantic Reconstruction of Continuous Language from Human Intracranial EEG

May 31, 2025

作者: Siavash Shams, Richard Antonello, Gavin Mischler, Stephan Bickel, Ashesh Mehta, Nima Mesgarani

cs.AI

摘要

解码神经信号中的连续语言仍然是神经科学与人工智能交叉领域的一项重大挑战。我们提出了Neuro2Semantic，一种新颖的框架，能够从颅内脑电图（iEEG）记录中重建感知语音的语义内容。我们的方法包含两个阶段：首先，基于LSTM的适配器将神经信号与预训练的文本嵌入对齐；其次，校正模块直接从这些对齐的嵌入中生成连续、自然的文本。这一灵活的方法克服了以往解码方法的局限，实现了无约束的文本生成。Neuro2Semantic仅需30分钟的神经数据即可实现强劲性能，在低数据环境下超越了近期的一项最先进方法。这些成果凸显了其在脑机接口和神经解码技术中实际应用的潜力。

English

Decoding continuous language from neural signals remains a significant challenge in the intersection of neuroscience and artificial intelligence. We introduce Neuro2Semantic, a novel framework that reconstructs the semantic content of perceived speech from intracranial EEG (iEEG) recordings. Our approach consists of two phases: first, an LSTM-based adapter aligns neural signals with pre-trained text embeddings; second, a corrector module generates continuous, natural text directly from these aligned embeddings. This flexible method overcomes the limitations of previous decoding approaches and enables unconstrained text generation. Neuro2Semantic achieves strong performance with as little as 30 minutes of neural data, outperforming a recent state-of-the-art method in low-data settings. These results highlight the potential for practical applications in brain-computer interfaces and neural decoding technologies.