通过测试时缩放实现跨语言推理

摘要

大型语言模型的推理能力研究主要集中在英语领域，即便这些预训练模型本身是多语言的。本研究探讨了基于长链思维（CoTs）的英语推理微调在多大程度上能够跨语言泛化。首先，我们发现，针对以英语为中心的推理语言模型（RLMs）增加推理计算规模，能够提升包括低资源语言在内的多种语言的数学推理能力，其表现甚至超越规模两倍于它们的模型。其次，我们揭示出，尽管以英语为中心的RLMs的CoTs自然以英语为主，但它们在处理引用的非英语输入时，始终遵循“引用-思考”模式进行推理。第三，我们找到了一种有效策略来控制长链CoT推理的语言，并观察到模型在高资源语言中推理更优且效率更高。最后，我们注意到，在跨领域推理泛化方面表现欠佳，特别是从STEM领域到文化常识知识的迁移，即便在英语中也是如此。总体而言，我们展示了英语推理测试时扩展的跨语言泛化潜力，研究了其机制，并勾勒了其局限性。我们得出结论，实践者应让以英语为中心的RLMs在高资源语言中进行推理，同时还需进一步工作以提升低资源语言及跨领域上下文中的推理能力。

English

Reasoning capabilities of large language models are primarily studied for English, even when pretrained models are multilingual. In this work, we investigate to what extent English reasoning finetuning with long chain-of-thoughts (CoTs) can generalize across languages. First, we find that scaling up inference compute for English-centric reasoning language models (RLMs) improves multilingual mathematical reasoning across many languages including low-resource languages, to an extent where they outperform models twice their size. Second, we reveal that while English-centric RLM's CoTs are naturally predominantly English, they consistently follow a quote-and-think pattern to reason about quoted non-English inputs. Third, we discover an effective strategy to control the language of long CoT reasoning, and we observe that models reason better and more efficiently in high-resource languages. Finally, we observe poor out-of-domain reasoning generalization, in particular from STEM to cultural commonsense knowledge, even for English. Overall, we demonstrate the potentials, study the mechanisms and outline the limitations of crosslingual generalization of English reasoning test-time scaling. We conclude that practitioners should let English-centric RLMs reason in high-resource languages, while further work is needed to improve reasoning in low-resource languages and out-of-domain contexts.

通过测试时缩放实现跨语言推理

Crosslingual Reasoning through Test-Time Scaling

摘要

Summary

Support

Support