透過有針對性的合成多語言資料使大型語言模型變得更聰明

摘要

語言特定能力（LSC）是指語言模型因提示所使用的語言不同，而在表現上出現較好或較差的現象。換言之，當以不同語言提示相同的語義查詢時，語言模型會產生不同（且可能錯誤）的回應。先前研究將此現象歸因於內部語義表徵在不同語言之間存在錯位。目前文獻中主要有兩種方法來解決 LSC：（1）將所有查詢統一經由英文處理，雖然能提升效能，但將語言表達力限制在英文；（2）在語言平衡的資料上進行訓練，雖然能均衡模型在不同語言上的效能，但會降低整體表現。在本研究中，我們從以資料為中心的視角出發，提出 HOTFIXR：針對提升跨語言推理的難度最佳化訓練資料（Hardness Optimized Training data For Improving X-Lingual Reasoning）。這是一個資料生成框架，利用模型來探查並學習學生模型的多語言弱點，進而生成資料以緩解這些弱點。HOTFIXR 能生成多語言合成訓練資料，從而提升多語言效能。我們在三個分佈內任務、三個分佈外任務以及四種分佈外語言上進行評估。平均而言，HOTFIXR（1）在分佈內效能上提升了 6.2%，（2）在分佈外任務上將微調所引發的災難性遺忘降低了 3.7%，（3）在分佈外語言上降低了 7.1%。整體而言，由於許多真實世界應用都需要多語言大型語言模型（LLM），我們的研究有助於讓 LLM 具備多語言能力的相關努力。我們將在論文被接受後釋出程式碼。

English

Language-specific competency (LSC) is the phenomenon of a language model performing better or worse depending on the language of the prompt. In other words, a language model outputs different (and potentially incorrect) responses to the same semantic query when prompted in different languages. Prior work attributes this to an internal misalignment of semantic representation across languages. Currently, there are two main approaches to address LSC in the literature: (1) routing all queries through English, improving performance, but limiting language expressivity to English; or (2) training on language-balanced data, equalizing model performance across languages, but reducing overall performance. In this work, we take a data centric perspective and introduce HOTFIXR: Hardness Optimized Training data For Improving X-Lingual Reasoning. It is a data generation framework that uses models to probe and learn a student model's multilingual weaknesses, and generates data to mitigate them. HOTFIXR can generate multilingual synthetic training data that can improve multilingual performance. We evaluate on three in-distribution tasks, three out-of-distribution tasks, and four out-of-distribution languages. On average, HOTFIXR (1) improves in-distribution performance by 6.2%, (2) reduces catastrophic forgetting (induced by fine-tuning) on OOD tasks by 3.7%, and (3) on OOD languages by 7.1%. Overall, as many real-world applications requires multilingual LLMs, our work contributes to the efforts of making LLMs multilingually proficient. We will release code upon acceptance.