AppAgentX：將GUI代理進化為熟練的智能手機用戶

摘要

近期大型語言模型（LLMs）的進展促成了基於LLM的智能代理的開發，這些代理能夠與圖形用戶界面（GUIs）進行互動。這些代理展現出強大的推理能力和適應性，使其能夠執行傳統上需要預定義規則的複雜任務。然而，基於LLM的代理依賴逐步推理，這往往導致效率低下，特別是在處理常規任務時。相比之下，傳統的基於規則的系統在效率上表現出色，但缺乏適應新場景的智能和靈活性。為了解決這一挑戰，我們提出了一種新穎的GUI代理進化框架，該框架在保持智能和靈活性的同時提升了操作效率。我們的方法引入了一種記憶機制，記錄代理的任務執行歷史。通過分析這些歷史，代理識別出重複的動作序列，並進化出高層次動作作為捷徑，取代這些低層次操作，從而提高效率。這使得代理能夠專注於需要更複雜推理的任務，同時簡化常規動作。在多個基準任務上的實驗結果表明，我們的方法在效率和準確性上均顯著優於現有方法。代碼將開源以支持進一步研究。

English

Recent advancements in Large Language Models (LLMs) have led to the development of intelligent LLM-based agents capable of interacting with graphical user interfaces (GUIs). These agents demonstrate strong reasoning and adaptability, enabling them to perform complex tasks that traditionally required predefined rules. However, the reliance on step-by-step reasoning in LLM-based agents often results in inefficiencies, particularly for routine tasks. In contrast, traditional rule-based systems excel in efficiency but lack the intelligence and flexibility to adapt to novel scenarios. To address this challenge, we propose a novel evolutionary framework for GUI agents that enhances operational efficiency while retaining intelligence and flexibility. Our approach incorporates a memory mechanism that records the agent's task execution history. By analyzing this history, the agent identifies repetitive action sequences and evolves high-level actions that act as shortcuts, replacing these low-level operations and improving efficiency. This allows the agent to focus on tasks requiring more complex reasoning, while simplifying routine actions. Experimental results on multiple benchmark tasks demonstrate that our approach significantly outperforms existing methods in both efficiency and accuracy. The code will be open-sourced to support further research.

AppAgentX：將GUI代理進化為熟練的智能手機用戶

AppAgentX: Evolving GUI Agents as Proficient Smartphone Users

摘要

Summary

Support

Support