ChatPaper.aiChatPaper

GUI-KV:基于时空感知KV缓存的高效GUI智能体

GUI-KV: Efficient GUI Agents via KV Cache with Spatio-Temporal Awareness

October 1, 2025
作者: Kung-Hsiang Huang, Haoyi Qiu, Yutong Dai, Caiming Xiong, Chien-Sheng Wu
cs.AI

摘要

基于视觉语言模型构建的图形用户界面(GUI)代理已成为自动化人机工作流程的一种前景广阔的方法。然而,这些代理在处理高分辨率截图序列和解决长期任务时也面临着效率挑战,导致推理速度慢、成本高且受限于内存。虽然键值(KV)缓存可以缓解这一问题,但在图像密集的场景中存储完整的缓存是不可行的。现有的缓存压缩方法并不理想,因为它们没有考虑到GUI的空间和时间冗余性。在本研究中,我们首先分析了GUI代理工作负载中的注意力模式,发现与自然图像不同,所有Transformer层中的注意力稀疏性均较高。这一洞察促使我们提出了一种简单的统一预算分配策略,实验表明该策略优于更复杂的层间变化方案。在此基础上,我们引入了GUI-KV,一种无需重新训练的即插即用KV缓存压缩方法。GUI-KV结合了两项新技术:(i)空间显著性引导,通过隐藏状态的L2范数增强注意力分数,以更好地保留语义重要的视觉标记;(ii)时间冗余评分,将前一帧的键投影到当前帧的键子空间,优先剪除冗余历史。在标准GUI代理基准测试和模型中,GUI-KV优于竞争性的KV压缩基线,在适度预算下与完整缓存的准确性非常接近。值得注意的是,在AgentNetBench基准测试的5张截图设置中,GUI-KV将解码浮点运算次数(FLOPs)减少了38.9%,同时将步骤准确性提高了4.1%。这些结果表明,利用GUI特有的冗余性可以实现高效且可靠的代理性能。
English
Graphical user interface (GUI) agents built on vision-language models have emerged as a promising approach to automate human-computer workflows. However, they also face the inefficiency challenge as they process long sequences of high-resolution screenshots and solving long-horizon tasks, making inference slow, costly and memory-bound. While key-value (KV) caching can mitigate this, storing the full cache is prohibitive for image-heavy contexts. Existing cache-compression methods are sub-optimal as they do not account for the spatial and temporal redundancy of GUIs. In this work, we first analyze attention patterns in GUI agent workloads and find that, unlike in natural images, attention sparsity is uniformly high across all transformer layers. This insight motivates a simple uniform budget allocation strategy, which we show empirically outperforms more complex layer-varying schemes. Building on this, we introduce GUI-KV, a plug-and-play KV cache compression method for GUI agents that requires no retraining. GUI-KV combines two novel techniques: (i) spatial saliency guidance, which augments attention scores with the L2 norm of hidden states to better preserve semantically important visual tokens, and (ii) temporal redundancy scoring, which projects previous frames' keys onto the current frame's key subspace to preferentially prune redundant history. Across standard GUI agent benchmarks and models, GUI-KV outperforms competitive KV compression baselines, closely matching full-cache accuracy at modest budgets. Notably, in a 5-screenshot setting on the AgentNetBench benchmark, GUI-KV reduces decoding FLOPs by 38.9% while increasing step accuracy by 4.1% over the full-cache baseline. These results demonstrate that exploiting GUI-specific redundancies enables efficient and reliable agent performance.
PDF62October 2, 2025