BrainExplore：大规模发现人脑中可解释的视觉表征

摘要

理解人类大脑如何表征视觉概念以及这些表征在哪些脑区编码，仍是一个长期存在的挑战。数十年的研究推进了我们对视觉表征的理解，但脑信号依然庞大而复杂，且可能的视觉概念空间极为广阔。因此，大多数研究仍停留在小规模阶段，依赖人工检查，聚焦于特定脑区和属性，且鲜少包含系统性验证。我们提出了一种大规模自动化框架，用于发现并解释人类大脑皮层中的视觉表征。我们的方法包含两个主要阶段：首先通过无监督数据驱动分解方法在功能磁共振成像活动中发现候选可解释模式；随后通过识别最能激发该模式的自然图像集，并生成描述其共享视觉含义的自然语言解释来阐释每个模式。为实现规模化处理，我们引入了自动化流程，可测试多个候选解释、分配定量可靠性评分，并为每个体素模式选择最一致的描述。该框架揭示了涵盖众多不同视觉概念的数千个可解释模式，包括此前未报道的细粒度表征。

English

Understanding how the human brain represents visual concepts, and in which brain regions these representations are encoded, remains a long-standing challenge. Decades of work have advanced our understanding of visual representations, yet brain signals remain large and complex, and the space of possible visual concepts is vast. As a result, most studies remain small-scale, rely on manual inspection, focus on specific regions and properties, and rarely include systematic validation. We present a large-scale, automated framework for discovering and explaining visual representations across the human cortex. Our method comprises two main stages. First, we discover candidate interpretable patterns in fMRI activity through unsupervised, data-driven decomposition methods. Next, we explain each pattern by identifying the set of natural images that most strongly elicit it and generating a natural-language description of their shared visual meaning. To scale this process, we introduce an automated pipeline that tests multiple candidate explanations, assigns quantitative reliability scores, and selects the most consistent description for each voxel pattern. Our framework reveals thousands of interpretable patterns spanning many distinct visual concepts, including fine-grained representations previously unreported.

BrainExplore：大规模发现人脑中可解释的视觉表征

BrainExplore: Large-Scale Discovery of Interpretable Visual Representations in the Human Brain

摘要

Support