BioMatrix：邁向覆蓋序列、結構與語言模態矩陣的全面生物基礎模型

摘要

我們提出 BioMatrix，這是首個在多模態基礎模型中，原生整合分子與蛋白質的序列、結構及自然語言，並採用單一解碼器架構的模型。現有的生物基礎模型分別追求原生多模態與廣泛的實體覆蓋：那些在共享目標下融合多種模態的模型，仍僅限於單一實體類型；而涵蓋多種實體類型的模型，則不是省略了明確的結構建模，就是依賴於適配器設計，使模型無法原生生成其所能讀取的模態。BioMatrix 透過統一分詞方案，將分子序列（支援 SMILES 與 SELFIES 表示法）、分子結構、蛋白質序列、蛋白質結構以及自然語言，映射至共享的離散詞元空間，使得所有模態能在單一的下一個詞元預測目標下，被一致地消費與生成——無需外部編碼器、投影適配器或模態特定的輸出頭。BioMatrix 建立在 Qwen3 語言模型（1.7B 與 4B）之上，在 3044 億個詞元上持續進行預訓練，涵蓋通用與領域特定文本、分子與蛋白質的序列與結構視角，以及跨模態語料庫——這些語料庫將生物分子實體與科學文本交織在一起，並透過分子-蛋白質與蛋白質-蛋白質交互作用數據連結不同實體。在針對涵蓋 6 大類別、共 80 項任務的下游應用綜合套件進行微調後——這些任務涵蓋跨模態與模態內的單實體與多實體理解及生成任務——BioMatrix 在 80 項任務中有 77 項達到最先進或具競爭力的表現，證明了單一、原生多模態的通用模型，能夠有效匹配甚至超越廣泛生物任務中的專門方法。

English

We present BioMatrix, the first multimodal foundation model that natively integrates sequences, structures, and natural language for both molecules and proteins within a single decoder-only architecture. Existing biological foundation models pursue native multimodality and broad entity coverage separately: those that fuse multiple modalities under a shared objective remain confined to a single entity type, while those spanning multiple entity types either omit explicit structural modeling or rely on adapter-based designs in which the model cannot natively generate the very modalities it can read. BioMatrix closes this gap by mapping molecular sequences (supporting both SMILES and SELFIES notations), molecular structures, protein sequences, protein structures, and natural language into a shared discrete token space through a unified tokenization scheme, so that all modalities are consumed and produced uniformly under a single next-token prediction objective -- without external encoders, projection adapters, or modality-specific output heads. Built upon the Qwen3 language model (1.7B and 4B), BioMatrix is continually pretrained on 304.4 billion tokens spanning general and domain-specific text, sequence and structure views of molecules and proteins, and cross-modal corpora that interleave biomolecular entities with scientific text and link distinct entities through molecule-protein and protein-protein interaction data. After tuning on a comprehensive suite of downstream applications covering 80 tasks across 6 categories -- encompassing single-entity and multi-entity understanding and generation tasks across and within modalities -- BioMatrix achieves state-of-the-art or competitive performance on 77 out of 80 tasks, demonstrating that a single, natively multimodal generalist model can effectively match or surpass specialized approaches across a wide range of biological tasks.