再思：大语言模型智能体行动与观察时的并行推理

摘要

在 ReAct 范式下，LLM 智能体交替进行推理、行动和观察，但有意识的推理仅限于思考阶段：当智能体串行执行动作并等待环境时，其推理则处于冻结状态。我们将行动与观察期间反复出现的这一间隔识别为推理空闲窗口，并探究它能否并行承载服务于未来轮次的额外推理。为此，我们提出了 Second Thought——一种无需训练的推理框架。它在每个思考阶段结束的瞬间分支出四个辅助分支，让它们与主循环并行解码，并在环境观察到达时将生成的思考结果合并回来。通过这种方式，Second Thought 将新增的推理从主线程的顺序解码路径上移出。在三个智能体基准和三个推理大语言模型上，Second Thought 在所有九个（模型，基准）组合中均降低了平均回合数，并在其中六个组合中减少了主线程解码量，最高减少 43%（在这些设置中平均约 20%），而在第七个组合中基本保持不变；Pass@1 在九个组合中的七个中没有显著变化，而两个显著差异分别为 +12.4 和 +10.2 个百分点。与一个计算量匹配的对照方法（该方法将等效的预算强制施加于主线程自身的推理）相比，在适用该对照的所有四个设置中，Second Thought 的 Pass@1 严格更高，同时顺序解码量减少 1.3 至 3.2。

English

LLM agents in the ReAct paradigm alternate between reasoning, acting, and observing, but deliberate reasoning is confined to the Thought phase: while the agent serializes an action and waits for the environment, its reasoning is frozen. We identify this recurring interval for Action and Observation as a reasoning idle window and ask whether it can host additional reasoning in parallel that serves future turns. Therefore, we propose Second Thought, a training-free inference framework that forks four auxiliary branches the instant each Thought phase concludes, decodes them concurrently with the main loop, and merges the generated thoughts back when the environment observation arrives. In this way, Second Thought relocates the added reasoning off the main thread's sequential decoding path. Across three agentic benchmarks and three reasoning LLMs, Second Thought lowers the average turn count in all nine (model,benchmark) pairs and reduces main thread decoding in six of them by up to 43% (roughly 20% on average among those settings), while leaving it essentially unchanged in a seventh; Pass@1 shows no significant change in seven of nine pairs and the two significant differences are +12.4 and +10.2 points. Against a compute-matched control that forces an equivalent budget onto the main thread's own reasoning, it attains strictly higher Pass@1 with 1.3 to 3.2 less sequential decoding in all four settings where the control applies.