透過對未見、低資源語言進行對比對齊指示來調整LLMs在機器翻譯中的應用

Tuning LLMs with Contrastive Alignment Instructions for Machine Translation in Unseen, Low-resource Languages

January 11, 2024

作者: Zhuoyuan Mao, Yen Yu

cs.AI

摘要

本文介紹對比對齊指示（AlignInstruct）以應對大型語言模型（LLMs）機器翻譯（MT）中的兩個挑戰。其中一個挑戰是將支持的語言擴展到以前未見過的語言。第二個挑戰與低資源語言中數據不足有關。通過MT指示（MTInstruct）對模型進行微調是應對第一個挑戰的一種直接方法。然而，MTInstruct受到第二個挑戰中固有的跨語言信號薄弱的限制。AlignInstruct通過使用統計詞對齊構建的跨語言鑑別器來強調跨語言監督。我們基於對BLOOMZ模型（1b1、3b和7b1）進行微調的結果顯示：（1）LLMs可以使用MTInstruct有效地翻譯未見過的語言；（2）AlignInstruct在涉及英語的48個翻譯方向上持續改善翻譯質量；（3）基於鑑別器的指示在跨語言指示中優於生成對應物；（4）AlignInstruct在30個零樣本方向中提高了性能。

English

This article introduces contrastive alignment instructions (AlignInstruct) to address two challenges in machine translation (MT) on large language models (LLMs). One is the expansion of supported languages to previously unseen ones. The second relates to the lack of data in low-resource languages. Model fine-tuning through MT instructions (MTInstruct) is a straightforward approach to the first challenge. However, MTInstruct is limited by weak cross-lingual signals inherent in the second challenge. AlignInstruct emphasizes cross-lingual supervision via a cross-lingual discriminator built using statistical word alignments. Our results based on fine-tuning the BLOOMZ models (1b1, 3b, and 7b1) in up to 24 unseen languages showed that: (1) LLMs can effectively translate unseen languages using MTInstruct; (2) AlignInstruct led to consistent improvements in translation quality across 48 translation directions involving English; (3) Discriminator-based instructions outperformed their generative counterparts as cross-lingual instructions; (4) AlignInstruct improved performance in 30 zero-shot directions.