就緒隊列：在LLM智能體控制中限制GPU機會並避免主機往返

摘要

LLM 代理服務在模型呼叫與工具呼叫之間反覆執行小規模的確定性轉換：路由結果、更新狀態，並發出下一個效果。我們探討這條控制路徑何時會展現出足夠的並行工作以供 GPU 執行，以及當 GPU 計算出的路由決策保留在裝置上時會有何不同。我們使用固定分割份額 F、精確離線份額 P*、區域性上限 U 與線上達成份額 A 來形式化就緒群組邊界。在零服務時間、無限容量與相等的相對啟動截止時間下，一項專用動態規劃可精確計算 P*。在對一個固定的 851 會話公共蹤跡面板進行平穩泊松重放時，主要條件在 100,000 個目標活躍會話、K=256 且啟動截止時間為 50 ms 下，得到 F=30.19%、P*=43.00% 與 U=45.85%。精確打包收回了固定視窗邊界所損失機會的 81.83%。由結果推導出的路由鍵是條件代理，而非可執行身分的證明。另一項機制研究將 GPU 計算出的二元決策保留在裝置上，而非將四個位元組傳回主機並重新分派。在四種指定的 GPU 放置方案中，裝置常駐路徑在所有 36 種配置下都更快；同一放置方案內的行中位數比率介於 1.19 倍到 2.39 倍之間。在兩種可接受的機制中，所有 14,557,440 次受測的批次呼叫都與單獨實作的主機預言機相符。一個不移除任何主機決策的固定巢狀裝置圖，在五種放置方案的所有 60 種配置中都較慢。整體而言，這些研究為 GPU 代理控制建立了兩個可量測的門檻：截止時間可行的群組供應量，以及觀測放置。要衡量 A、CPU 置換與服務層級效益，需要一個整合式的有限線上執行期。

English

LLM-agent services repeatedly execute small deterministic transitions between model and tool calls: route an outcome, update state, and emit the next effect. We ask when this control path exposes enough concurrent work for GPU execution, and what changes when a GPU-computed route decision remains on device. We formalize the ready-cohort boundary using fixed-partition share F, exact offline share P*, local upper bound U, and online achieved share A. Under zero service time, unlimited capacity, and equal relative launch deadlines, a specialized dynamic program computes P* exactly. In a stationary Poisson replay of one pinned 851-session public trace panel, the primary condition at 100,000 target active sessions, K=256, and a 50 ms launch deadline gives F=30.19%, P*=43.00%, and U=45.85%. Exact packing recovers 81.83% of the opportunity lost at fixed window boundaries. The outcome-derived route key is a conditioning proxy, not proof of executable identity. A separate mechanism study keeps a GPU-computed binary decision on device instead of returning four bytes to the host and redispatching. Across four named GPU placements, the device-resident path is faster in all 36 configurations; within-placement row-median ratios range from 1.19x to 2.39x. Across both admissible mechanisms, all 14,557,440 tested batched invocations match a separately implemented host oracle. A fixed nested device graph that removes no host decision is slower in all 60 configurations across five placements. Together, the studies establish two measurable gates for GPU agent control: deadline-feasible cohort supply and observation placement. A joined finite online runtime is required to measure A, CPU displacement, and service-level benefit.