解码轨迹辅助的LLM推理：优化视角

Deciphering Trajectory-Aided LLM Reasoning: An Optimization Perspective

May 26, 2025

作者: Junnan Liu, Hongwei Liu, Linchen Xiao, Shudong Liu, Taolin Zhang, Zihan Ma, Songyang Zhang, Kai Chen

cs.AI

摘要

我们提出了一种新颖的框架，通过元学习的视角来理解大语言模型（LLMs）的推理能力。通过将推理轨迹概念化为对LLM参数的伪梯度下降更新，我们识别出LLM推理与多种元学习范式之间的相似性。我们将推理任务的训练过程形式化为一个元学习设置，其中每个问题被视为一个独立任务，而推理轨迹则作为适应模型参数的内循环优化。一旦在多样化的问题集上完成训练，LLM便能发展出可推广到未见问题的基础推理能力。大量的实证评估证实了LLM推理与元学习之间的紧密联系，从元学习的角度探讨了多个具有重要意义的问题。我们的工作不仅加深了对LLM推理的理解，还通过成熟的元学习技术为改进这些模型提供了实用的见解。

English

We propose a novel framework for comprehending the reasoning capabilities of large language models (LLMs) through the perspective of meta-learning. By conceptualizing reasoning trajectories as pseudo-gradient descent updates to the LLM's parameters, we identify parallels between LLM reasoning and various meta-learning paradigms. We formalize the training process for reasoning tasks as a meta-learning setup, with each question treated as an individual task, and reasoning trajectories serving as the inner loop optimization for adapting model parameters. Once trained on a diverse set of questions, the LLM develops fundamental reasoning capabilities that can generalize to previously unseen questions. Extensive empirical evaluations substantiate the strong connection between LLM reasoning and meta-learning, exploring several issues of significant interest from a meta-learning standpoint. Our work not only enhances the understanding of LLM reasoning but also provides practical insights for improving these models through established meta-learning techniques.