用于音节标记化的说话人解耦逐块回归

摘要

无监督音节标记化旨在从原始语音中学习离散的音节标记，以捕捉与潜在语言内容相关的结构。最近的音节标记化方法采用预训练HuBERT的师生蒸馏，将潜在的语音帧表示组织成音节片段。然而，当使用话语级交叉熵目标进行训练时，模型会预测说话人身份而非语言内容，从而损害音节标记的纯度。为解决这一问题，我们提出了一种说话人解耦的音节标记器，该标记器在固定长度块内将受说话人扰动的学生表示回归至干净的教师目标。实验结果表明，我们的方法在音节边界检测和音节片段聚类方面达到了最先进的性能。此外，基于我们的音节标记训练的语音语言模型，在句法和语义理解上相比于音素级别的SpiRit-LM取得了7%的相对提升。

English

Unsupervised syllabic tokenization aims to learn discrete syllabic tokens that capture latent linguistic content-related structure from raw speech. Recent syllabic tokenization methods employ teacher-student distillation of the pretrained HuBERT to organize latent speech frame representations into syllabic segments. However, when trained with an utterance-level cross-entropy objective, the model predicts speaker identity rather than linguistic content, thereby compromising the purity of syllabic tokens. To address this problem, we propose a speaker-disentangled syllabic tokenizer that regresses speaker-perturbed student representations toward clean teacher targets within fixed-length chunks. Experimental results demonstrate that our proposed method achieves state-of-the-art performance in syllable boundary detection and syllabic segment clustering. Moreover, a speech language model trained on our syllabic tokens achieves a 7% relative improvement in syntactic and semantic understanding over the phone-level SpiRit-LM.