三思而行：大型語言模型智能體在行動與觀察中的並行推理

摘要

在 ReAct 範式中，LLM 智能體在推理、行動與觀察之間交替進行，但審慎的推理僅限於「思考」階段：當智能體序列化一個行動並等待環境回應時，其推理便陷入凍結。我們將這段在行動與觀察期間反覆出現的間隔視為一個推理閒置窗口，並探討它能否並行地承載額外的推理，以服務於未來的回合。為此，我們提出了 Second Thought——一個免訓練的推論框架；它在每個思考階段結束的當下即刻分出四個輔助分支，與主迴圈並行解碼，並在環境觀察到達時將這些生成的想法合併回來。如此一來，Second Thought 便將新增的推理移出主執行緒的序列化解碼路徑。在三個智能體基準測試與三個推理型 LLM 上，Second Thought 於全部九個（模型，基準）組合中降低了平均回合數，並在其中六個組合中將主執行緒解碼量減少最多 43%（這些設定下平均約 20%），在第七個組合中則基本上未變；Pass@1 在九組中的七組沒有顯著變化，而兩個顯著差異分別為 +12.4 與 +10.2 個百分點。與計算量匹配的對照組（該對照組將等量預算強加於主執行緒自身的推理）相比，在對照組適用的所有四個設定中，Second Thought 皆以少 1.3 至 3.2 的序列化解碼量取得了嚴格更高的 Pass@1。

English

LLM agents in the ReAct paradigm alternate between reasoning, acting, and observing, but deliberate reasoning is confined to the Thought phase: while the agent serializes an action and waits for the environment, its reasoning is frozen. We identify this recurring interval for Action and Observation as a reasoning idle window and ask whether it can host additional reasoning in parallel that serves future turns. Therefore, we propose Second Thought, a training-free inference framework that forks four auxiliary branches the instant each Thought phase concludes, decodes them concurrently with the main loop, and merges the generated thoughts back when the environment observation arrives. In this way, Second Thought relocates the added reasoning off the main thread's sequential decoding path. Across three agentic benchmarks and three reasoning LLMs, Second Thought lowers the average turn count in all nine (model,benchmark) pairs and reduces main thread decoding in six of them by up to 43% (roughly 20% on average among those settings), while leaving it essentially unchanged in a seventh; Pass@1 shows no significant change in seven of nine pairs and the two significant differences are +12.4 and +10.2 points. Against a compute-matched control that forces an equivalent budget onto the main thread's own reasoning, it attains strictly higher Pass@1 with 1.3 to 3.2 less sequential decoding in all four settings where the control applies.