在語音模型中進行大寫和輪替預測的文本注入

摘要

對於自動語音識別（ASR）的文本注入，即使用未配對的僅文本數據來補充配對的音頻-文本數據，已經顯示出對於詞錯率有著令人期待的改進。本研究探討了文本注入用於輔助任務，這些任務通常由端對端（E2E）模型執行。在這項工作中，我們使用聯合端對端和內部語言模型訓練（JEIT）作為我們的文本注入算法，來訓練一個ASR模型，該模型執行兩個輔助任務。第一個是大寫化，這是一個去正規化的任務。第二個是轉換預測，該任務試圖識別用戶是否已完成他們在數字助理互動中的對話輪。我們展示了結果，證明了我們的文本注入方法提升了長尾數據的大寫化性能，並改善了轉換檢測的召回率。

English

Text injection for automatic speech recognition (ASR), wherein unpaired text-only data is used to supplement paired audio-text data, has shown promising improvements for word error rate. This study examines the use of text injection for auxiliary tasks, which are the non-ASR tasks often performed by an E2E model. In this work, we use joint end-to-end and internal language model training (JEIT) as our text injection algorithm to train an ASR model which performs two auxiliary tasks. The first is capitalization, which is a de-normalization task. The second is turn-taking prediction, which attempts to identify whether a user has completed their conversation turn in a digital assistant interaction. We show results demonstrating that our text injection method boosts capitalization performance for long-tail data, and improves turn-taking detection recall.

在語音模型中進行大寫和輪替預測的文本注入

Text Injection for Capitalization and Turn-Taking Prediction in Speech Models

摘要

Support