在GPT中解析上下文學習翻譯

Dissecting In-Context Learning of Translations in GPTs

October 24, 2023

作者: Vikas Raunak, Hany Hassan Awadalla, Arul Menezes

cs.AI

摘要

近期大部分關於利用大型語言模型（LLMs）如GPT-3進行機器翻譯（MT）的研究，著重於選擇少量樣本進行提示。在這項研究中，我們試圖透過對高質量、領域內示範的干擾，更好地理解示範屬性在上下文學習翻譯中的作用。我們發現對源-目標映射進行非對稱干擾會產生截然不同的結果。我們展示了對源端進行干擾對結果影響微乎其微，而對目標進行干擾則會顯著降低翻譯質量，這表明輸出文本分佈提供了最重要的學習信號，用於上下文學習翻譯。我們提出了一種名為Zero-Shot-Context的方法，自動在零提示中添加這個信號。我們證明它提升了GPT-3的零提示翻譯性能，甚至使其與少提示翻譯相競爭。

English

Most of the recent work in leveraging Large Language Models (LLMs) such as GPT-3 for Machine Translation (MT) has focused on selecting the few-shot samples for prompting. In this work, we try to better understand the role of demonstration attributes for the in-context learning of translations through perturbations of high-quality, in-domain demonstrations. We find that asymmetric perturbation of the source-target mappings yield vastly different results. We show that the perturbation of the source side has surprisingly little impact, while target perturbation can drastically reduce translation quality, suggesting that it is the output text distribution that provides the most important learning signal during in-context learning of translations. We propose a method named Zero-Shot-Context to add this signal automatically in Zero-Shot prompting. We demonstrate that it improves upon the zero-shot translation performance of GPT-3, even making it competitive with few-shot prompted translations.