朝向適用於裝置虛擬助理的全球英語語言模型

Towards a World-English Language Model for On-Device Virtual Assistants

March 27, 2024

作者: Rricha Jalota, Lyan Verwimp, Markus Nussbaum-Thom, Amr Mousa, Arturo Argueta, Youssef Oualil

cs.AI

摘要

神經網絡語言模型（NNLMs）用於虛擬助手（VAs）通常依賴於語言、地區，有時也依賴於設備，這增加了擴展和維護的工作量。結合一個或多個類別的NNLMs是提高可擴展性的一種方法。在這項工作中，我們結合了英語的地區變體，建立了一個“世界英語”NNLM，用於設備上的VAs。具體來說，我們研究了適配器瓶頸的應用，以模擬我們現有生產的NNLMs中的方言特徵，並增強多方言基準。我們發現適配器模塊在模擬方言方面比專門化整個子網絡更有效。基於這一見解，並利用我們生產模型的設計，我們引入了一種新的架構，用於世界英語NNLM，滿足我們單方言模型的準確性、延遲和內存限制。

English

Neural Network Language Models (NNLMs) for Virtual Assistants (VAs) are generally language-, region-, and in some cases, device-dependent, which increases the effort to scale and maintain them. Combining NNLMs for one or more of the categories is one way to improve scalability. In this work, we combine regional variants of English to build a ``World English'' NNLM for on-device VAs. In particular, we investigate the application of adapter bottlenecks to model dialect-specific characteristics in our existing production NNLMs {and enhance the multi-dialect baselines}. We find that adapter modules are more effective in modeling dialects than specializing entire sub-networks. Based on this insight and leveraging the design of our production models, we introduce a new architecture for World English NNLM that meets the accuracy, latency, and memory constraints of our single-dialect models.