FlashRT: 用于指导智能体部署实时多模态应用的智能体框架

摘要

实时多模态应用（包括语音交互智能体和交互式视频生成）将异构模型组合成流水线，其高效部署需要针对应用场景制定关于模型放置、数据流传输和模型内并行策略的定制化决策。现有的服务系统和自动并行编译器局限于有限的转换方式和固定的负载假设，因此在新应用中实现高性能通常需要手动打造高效的实现方案。本文提出FlashRT——一种智能体框架，该框架引导代码生成智能体将开发者编写的简单参考实现转化为优化的多GPU部署方案，可灵活权衡延迟、吞吐量等目标指标。通过创新的"编程链"范式，FlashRT引导通用代码生成智能体经历多阶段转换流程：智能体首先将参考实现转换为中间表示（IR）以捕获数据依赖关系和持久状态作用域，接着通过顺序解释器验证该IR，并执行静态分析以识别候选转换策略。随后，智能体在测量驱动的优化循环中迭代实施、验证并基准测试每个候选方案，最终生成适配不同硬件预算的高效部署方案。在包括视频世界模型和多模态大语言模型在内的多种应用中，FlashRT将参考实现转化为高效部署方案，在NVIDIA B200 GPU上实现最高约70倍延迟降低和2.8倍吞吐量提升。在AMD MI355X GPU上，FlashRT在保持最大延迟降低效果的同时，将峰值吞吐量提升至3.6倍，表明智能体驱动的优化在专家优化较不成熟的平台上展现出更强的可扩展性。事实上，在Qwen3-Omni文本转语音推理任务中，与AMD MI355X平台上的专家级vLLM-Omni实现相比，FlashRT将响应延迟降低了65%。

English

Real-time multimodal applications, including voice agents and interactive video generation, compose heterogeneous models into pipelines whose efficient deployment requires application-specific decisions about placement, streaming, and intra-model parallelism. Existing serving systems and auto-parallelism compilers commit to limited transformations and fixed workload assumptions, so achieving high performance on a new application requires hand-crafting an efficient implementation. We present FlashRT, an agent harness that guides coding agents to lift simple developer-written reference implementations into optimized multi-GPU deployments that flexibly weigh target metrics like latency and throughput. Using a new chain-of-program paradigm, FlashRT directs a generic coding agent through a multi-pass transformation process where an agent transforms the reference into an intermediate representation (IR) to capture data dependencies and persistent-state scopes, validates this IR via a sequential interpreter, and performs static analyses to identify candidate transformations. Then, the agent iteratively implements, verifies, and benchmarks each candidate under a measurement-gated optimization loop to produce effective deployments that span different hardware budgets. Across various applications, including video world models and multimodal LLMs, FlashRT converts reference implementations into highly efficient deployments, delivering up to ~70x latency reduction and 2.8x throughput improvement on NVIDIA B200 GPUs. On AMD MI355X GPUs, FlashRT matches the peak latency reduction while increasing peak throughput improvement to 3.6x, demonstrating that agent-driven optimization can be more scalable on platforms with less mature expert optimization. In fact, for Qwen3-Omni text-to-audio inference, FlashRT reduces response latency by 65% compared to the expert vLLM-Omni implementation on AMD MI355X.