雙層協同反思：多智能體大語言模型系統的博弈論方法

摘要

多代理大型語言模型系統通常使用一個編排者將任務分解給一組工作者，然後透過文字反思持續改進。儘管經驗結果強勁，此類系統在協調、記憶改進以及外部驗證的角色上仍缺乏統一的詮釋。我們將編排者與工作者之間的互動建模為雙層協調賽局：在有界耦合下，工作者的局部更新賽局是一個近似勢能賽局，其均衡鬆弛度由分解品質控制。接著，我們將反思分析為作用於語義記憶狀態上的隨機移動。針對自由形式的反思，我們推導出有限時間上界，證明最壞情況下的緊緻性，並在可證偽的持續危害條件下給出正下界。我們進一步證明一個資訊理論上的不可能結果：任何僅觀察所生成文本的門控，皆無法在文本不可區分的環境上一致地改善；反之，基於環境的門控則可以。受此區別啟發，我們提出隨機反思記憶上升法（SRMA），該方法僅在基於環境的評估風險嚴格下降後，才接受候選記憶。在校準與非退化修正質量的條件下，SRMA 精確收斂，且收斂速率為幾何級數或多項式級數；配對構造顯示這兩個速率區間在階的意義下皆為緊緻。我們亦針對隨機評估提供信心門控，並為分段平穩環境提供重新錨定保證。實驗以基於環境的指標實例化上述物件，並檢驗所預測的協調與漂移定律。在 500 個 SWE-bench 實例中，完整的 Kimi 基礎系統解決了 72.2%，而公開的 mini-SWE-agent 參考系統則為 70.8%。程式碼：https://github.com/YihangChen9/Bilevel-Coordinated-Reflection

English

Multi-agent LLM systems commonly use an orchestrator to decompose a task for a team of workers and then improve through textual reflection. Despite strong empirical results, these systems lack a unified account of coordination, memory improvement, and the role of external verification. We model orchestrator-worker interaction as a bilevel coordination game: under bounded coupling, the workers' local-update game is an approximate potential game whose equilibrium slack is controlled by decomposition quality. We then analyse reflection as stochastic movement over semantic memory states. For free-form reflection, we derive a finite-time upper bound, prove worst-case tightness, and give a positive lower bound under a falsifiable persistent-harm condition. We further prove an information-theoretic impossibility result: no gate that observes only the generated transcript can improve uniformly over text-indistinguishable environments, whereas an environment-grounded gate can. Motivated by this separation, we introduce Stochastic Reflective Memory Ascent (SRMA), which accepts a candidate memory only after a grounded evaluation risk strictly decreases. Under calibration and non-degenerate corrective mass, SRMA converges exactly, geometrically or polynomially; matching constructions show that both rate regimes are order-tight. We also provide confidence gating for stochastic evaluation and re-anchoring guarantees for piecewise-stationary environments. Experiments instantiate these objects with environment-grounded metrics and test the predicted coordination and drift laws. On 500 SWE-bench instances, the complete Kimi-based system resolves 72.2% versus a 70.8% public mini-SWE-agent reference. Code: https://github.com/YihangChen9/Bilevel-Coordinated-Reflection