实现面向设备的虚拟助手的全球英语语言模型

Towards a World-English Language Model for On-Device Virtual Assistants

March 27, 2024

作者: Rricha Jalota, Lyan Verwimp, Markus Nussbaum-Thom, Amr Mousa, Arturo Argueta, Youssef Oualil

cs.AI

摘要

神经网络语言模型（NNLMs）用于虚拟助手（VAs）通常是依赖于语言、地区，有时还依赖于设备，这增加了扩展和维护的工作量。将NNLMs结合用于一个或多个类别是提高可扩展性的一种方法。在这项工作中，我们结合了英语的地区变体，构建了一个用于设备上的VAs的“世界英语”NNLM。具体来说，我们研究了适配器瓶颈的应用，以模拟我们现有生产NNLMs中的方言特征，并增强多方言基线。我们发现适配器模块在模拟方言方面比专门化整个子网络更有效。基于这一观点，并利用我们生产模型的设计，我们引入了一种新的架构，用于世界英语NNLM，满足我们单一方言模型的准确性、延迟和内存约束。

English

Neural Network Language Models (NNLMs) for Virtual Assistants (VAs) are generally language-, region-, and in some cases, device-dependent, which increases the effort to scale and maintain them. Combining NNLMs for one or more of the categories is one way to improve scalability. In this work, we combine regional variants of English to build a ``World English'' NNLM for on-device VAs. In particular, we investigate the application of adapter bottlenecks to model dialect-specific characteristics in our existing production NNLMs {and enhance the multi-dialect baselines}. We find that adapter modules are more effective in modeling dialects than specializing entire sub-networks. Based on this insight and leveraging the design of our production models, we introduce a new architecture for World English NNLM that meets the accuracy, latency, and memory constraints of our single-dialect models.