VibeVoice流式语音识别技术报告

摘要

传统的说话人归属语音识别（speaker-attributed ASR）系统将语音识别和说话人日志（speaker diarization）视为两个独立的任务。近年来，VibeVoice-ASR 等端到端模型已将这两个任务统一到单一模型中。然而，现有的统一模型仍主要支持离线识别，难以满足实时语音助手和智能体的低延迟需求。为解决这一问题，我们提出了 VibeVoice-ASR-Streaming，这是首批基于大语言模型（LLM）的流式说话人归属语音识别端到端方法之一。该方法将固定大小的音频块、少量前瞻音频和先前文本交错输入，使模型能够在语音到达时实时输出"谁说了什么"，而无需单独的说话人日志阶段。在转录准确率方面，我们的 7B 模型在五个评估集上取得了最低的平均词错误率（WER）/字符错误率（CER）。在说话人归属方面，该模型在 13 个评估设置中的 12 个上达到最佳或并列最佳。我们开源了 1.5B 和 7B 模型的权重以及推理代码。

English

Traditional speaker-attributed ASR systems treated ASR and speaker diarization as two separate tasks. Recently, end-to-end models such as VibeVoice-ASR have unified the two tasks within a single model. However, existing unified models still mainly support offline recognition, making it difficult to meet the low-latency requirements of real-time voice assistants and agents. To tackle this issue, we present VibeVoice-ASR-Streaming, one of the first LLM-based end-to-end approaches to streaming speaker-attributed ASR. It interleaves fixed-size audio chunks, a small amount of lookahead audio and previous text. This allows the model to produce ''who said what'' as speech arrives, without a separate diarization stage. For transcription accuracy, our 7B model achieves the lowest average WER/CER across five evaluation sets. For speaker attribution, it achieves the best or tied-best on 12 of 13 evaluation settings. We release the 1.5B and 7B model weights together with inference code.