大语言模型从定向合成多语言数据中变得更聪明

摘要

语言特定能力（LSC）是指语言模型根据提示所用语言的不同而表现出更好或更差性能的现象。换言之，当使用不同语言提示相同的语义查询时，语言模型会输出不同（且可能不正确）的响应。先前工作将其归因于跨语言语义表征的内部错位。目前，文献中解决LSC的主要方法有两种：（1）将所有查询路由到英语，从而提升性能，但将语言表达能力限制为英语；（2）在语言平衡数据上训练，使模型性能在各语言间均衡，但会降低整体性能。在本工作中，我们采取以数据为中心的视角，引入了HOTFIXR：面向提升跨语言推理的难度优化训练数据生成框架。该框架利用模型探测并学习学生模型的多语言弱点，并生成数据以缓解这些弱点。HOTFIXR能够生成多语言合成训练数据，从而提升多语言性能。我们在三个分布内任务、三个分布外任务和四种分布外语言上进行了评估。平均而言，HOTFIXR（1）将分布内性能提升6.2%；（2）在分布外任务上将微调引发的灾难性遗忘降低3.7%；（3）在分布外语言上将灾难性遗忘降低7.1%。总体而言，由于许多现实世界应用需要多语言LLM，我们的工作为提升LLM的多语言熟练度做出了贡献。我们将于论文被接收后发布代码。

English

Language-specific competency (LSC) is the phenomenon of a language model performing better or worse depending on the language of the prompt. In other words, a language model outputs different (and potentially incorrect) responses to the same semantic query when prompted in different languages. Prior work attributes this to an internal misalignment of semantic representation across languages. Currently, there are two main approaches to address LSC in the literature: (1) routing all queries through English, improving performance, but limiting language expressivity to English; or (2) training on language-balanced data, equalizing model performance across languages, but reducing overall performance. In this work, we take a data centric perspective and introduce HOTFIXR: Hardness Optimized Training data For Improving X-Lingual Reasoning. It is a data generation framework that uses models to probe and learn a student model's multilingual weaknesses, and generates data to mitigate them. HOTFIXR can generate multilingual synthetic training data that can improve multilingual performance. We evaluate on three in-distribution tasks, three out-of-distribution tasks, and four out-of-distribution languages. On average, HOTFIXR (1) improves in-distribution performance by 6.2%, (2) reduces catastrophic forgetting (induced by fine-tuning) on OOD tasks by 3.7%, and (3) on OOD languages by 7.1%. Overall, as many real-world applications requires multilingual LLMs, our work contributes to the efforts of making LLMs multilingually proficient. We will release code upon acceptance.