就绪队列组：在LLM智能体控制中约束GPU机会并避免主机往返

摘要

LLM智能体服务在模型调用与工具调用之间反复执行小型确定性状态转换：对结果进行路由、更新状态并发出下一个效应。我们探究该控制路径在何种条件下能为GPU执行暴露足够的并发工作，以及当由GPU计算的路由决策保留在设备上时会发生什么变化。我们使用固定分区份额F、精确离线份额P*、局部上界U和在线达成份额A来形式化就绪队列边界。在零服务时间、无限容量和相同相对启动截止时间的条件下，一个专门的动态规划程序可精确计算P*。在对一个固定的851会话公共轨迹面板进行平稳泊松重放时，在100,000个目标活跃会话、K=256和50毫秒启动截止时间的主要条件下，F=30.19%，P*=43.00%，U=45.85%。精确打包恢复了因固定窗口边界而损失的81.83%的机会。由结果派生的路由键是条件化代理，而非可执行身份的证明。一项独立的机制研究将GPU计算的二元决策保留在设备上，而不是向主机返回四个字节并重新调度。在四个命名的GPU放置方案中，设备驻留路径在所有36种配置中均更快；放置方案内的行中位数比率范围为1.19倍至2.39倍。在两种可接受的机制中，所有14,557,440次测试的批量调用均与单独实现的主机预言机匹配。一个不移除任何主机决策的固定嵌套设备图在五种放置方案的所有60种配置中均较慢。综合来看，这些研究确立了GPU智能体控制的两个可测量门控：满足截止时间的队列供给和观测放置。需要一个联合有限在线运行时来测量A、CPU置换和服务级收益。

English

LLM-agent services repeatedly execute small deterministic transitions between model and tool calls: route an outcome, update state, and emit the next effect. We ask when this control path exposes enough concurrent work for GPU execution, and what changes when a GPU-computed route decision remains on device. We formalize the ready-cohort boundary using fixed-partition share F, exact offline share P*, local upper bound U, and online achieved share A. Under zero service time, unlimited capacity, and equal relative launch deadlines, a specialized dynamic program computes P* exactly. In a stationary Poisson replay of one pinned 851-session public trace panel, the primary condition at 100,000 target active sessions, K=256, and a 50 ms launch deadline gives F=30.19%, P*=43.00%, and U=45.85%. Exact packing recovers 81.83% of the opportunity lost at fixed window boundaries. The outcome-derived route key is a conditioning proxy, not proof of executable identity. A separate mechanism study keeps a GPU-computed binary decision on device instead of returning four bytes to the host and redispatching. Across four named GPU placements, the device-resident path is faster in all 36 configurations; within-placement row-median ratios range from 1.19x to 2.39x. Across both admissible mechanisms, all 14,557,440 tested batched invocations match a separately implemented host oracle. A fixed nested device graph that removes no host decision is slower in all 60 configurations across five placements. Together, the studies establish two measurable gates for GPU agent control: deadline-feasible cohort supply and observation placement. A joined finite online runtime is required to measure A, CPU displacement, and service-level benefit.