Neuro2Semantic：一种用于人类颅内脑电图连续语言语义重建的迁移学习框架

摘要

從神經信號解碼連續語言仍是神經科學與人工智能交叉領域中的一項重大挑戰。我們提出了Neuro2Semantic，這是一種新穎的框架，能夠從顱內腦電圖（iEEG）記錄中重建感知語言的語義內容。我們的方法分為兩個階段：首先，基於LSTM的適配器將神經信號與預訓練的文本嵌入對齊；其次，校正模塊直接從這些對齊的嵌入生成連續、自然的文本。這種靈活的方法克服了以往解碼方法的限制，實現了無約束的文本生成。Neuro2Semantic僅需30分鐘的神經數據即可展現出強大的性能，在低數據設置下超越了近期的一種最先進方法。這些結果凸顯了其在腦機接口和神經解碼技術中實際應用的潛力。

English

Decoding continuous language from neural signals remains a significant challenge in the intersection of neuroscience and artificial intelligence. We introduce Neuro2Semantic, a novel framework that reconstructs the semantic content of perceived speech from intracranial EEG (iEEG) recordings. Our approach consists of two phases: first, an LSTM-based adapter aligns neural signals with pre-trained text embeddings; second, a corrector module generates continuous, natural text directly from these aligned embeddings. This flexible method overcomes the limitations of previous decoding approaches and enables unconstrained text generation. Neuro2Semantic achieves strong performance with as little as 30 minutes of neural data, outperforming a recent state-of-the-art method in low-data settings. These results highlight the potential for practical applications in brain-computer interfaces and neural decoding technologies.

Neuro2Semantic：一种用于人类颅内脑电图连续语言语义重建的迁移学习框架

Neuro2Semantic: A Transfer Learning Framework for Semantic Reconstruction of Continuous Language from Human Intracranial EEG

摘要

Support