何处切割，深度几何：BPE与Unigram-LM在化学SMILES上的应用

摘要

每一个读取SMILES的化学语言模型都从分词器开始，然而该领域几乎未经审视地沿用了自然语言处理中的字节对编码（BPE）。在自然语言中，BPE的主要替代方案Unigram-LM能够构建结构上不同的词汇表。这种差异在化学领域是否依然存在尚不明确。我们报告了在固定165个化学基元的小词汇表规模下（使得词嵌入可学习），对BPE与Unigram-LM进行的控制比较实验。实验覆盖三种语料类型（多样性、类药性、天然产物）和两种预分词边界策略。两者并未收敛。在所有22组匹配条件下，它们构建了近乎不相交的子词词汇表：跨算法的学习片段Jaccard重叠度从未超过0.161，若按模型更新最频繁的高频片段加权，则至多为0.05。Unigram-LM还将保留分子分割为多29%-41%的词元；两种算法在切割位置上大致一致，但切割深度不同——在80%-99%的分子上，BPE的分割严格是Unigram-LM的粗化版本。这种差异在语料、边界和词汇表规模上均保持稳定，甚至在词汇表扩大到八倍时依然存在。因此，子词算法是一项建模决策，而非可随意使用的默认选项。本研究未训练任何语言模型。

English

Every chemical language model reading SMILES begins with a tokenizer, yet the field has inherited byte-pair encoding (BPE) from natural language with little scrutiny. In natural language, BPE's principal alternative, Unigram-LM, is known to build structurally different vocabularies. Whether that contrast survives in chemistry was open. We report a controlled comparison of BPE and Unigram-LM over a fixed 165-token chemistry base, at the small vocabulary sizes where token embeddings are learnable, across three corpus typologies (diverse, drug-like, natural-products) and both pre-tokenization boundary policies. The two do not converge. In all 22 matched conditions they build near-disjoint subword vocabularies: cross-algorithm Jaccard overlap on the learned pieces never exceeds 0.161, and at most 0.05 once weighted toward the high-frequency pieces a model updates most. Unigram-LM also segments held-out molecules into 29-41% more tokens; the arms largely agree on where to cut but not how deeply, so BPE's segmentation is a strict coarsening of Unigram-LM's on 80-99% of molecules. The separation holds across corpus, boundary, and vocabulary size, persisting even at eight times that scale. The subword algorithm is therefore a modeling decision, not a free default. The study trains no language models.