Qwen-UI-Agent技术报告：迈向下一代以真实世界为核心的基础图形用户界面智能体

摘要

GUI代理有潜力成为现有数字设备上的通用执行器。为推动其走向实际应用，我们设想这样的代理：能在真实设备上可靠运行、跨平台执行工作流、将GUI交互与命令行执行相结合、完成长时程任务、主动发起有用的服务，并以最少的人力投入自主提升能力。在这一愿景的指引下，我们提出Qwen-UI-Agent——一个以真实世界为核心的基座GUI代理，覆盖移动端、电脑操作、网页及DeepSearch环境。Qwen-UI-Agent将多样化的沙箱环境与大规模真实设备移动运行时相结合。其统一动作空间将GUI操作与命令行执行交织在一起，并在单次模型生成中输出批量动作。一个AutoResearch风格的数据飞轮利用代理来构建任务与环境、诊断失败并规划后续迭代。在线强化学习支持对超过100轮轨迹的训练，超过10,000个并发环境加速数据采集。一个轻量级封装层支持跨移动端和电脑的主动服务发起及有状态工作流。在广泛的评测套件中，Qwen-UI-Agent在移动端使用基准上取得了最先进的性能，同时在电脑端和浏览器端任务上与前沿模型（包括Opus 4.8、Gemini 3.1 Pro和GPT-5.6 Sol）相比展现了具有竞争力的表现。在移动端使用中，其在MobileWorld上达到82.1%，在MobileWorld-Real上达到92.2%，在AndroidDaily上达到97.5%。在电脑端使用中，其在OSWorld-Verified上达到79.5%，在OSWorld-v2上取得40.0%的部分进展分数。在浏览器端使用和GUI定位方面，其分别在WebArena上达到73.6%，在ScreenSpot-Pro上达到81.5%。

English

GUI agents have the potential to become a general purpose executor over existing digital devices. To advance them toward real-world use, we envision agents that operate reliably on real devices, execute workflows across platforms, combine GUI interaction with CLI execution, complete long-horizon tasks, proactively initiate useful services, and autonomously improve their capabilities with minimal human effort. Guided by this vision, we present Qwen-UI-Agent, a real-world centric foundation GUI agent spanning mobile, computer-use, web, and DeepSearch environments. Qwen-UI-Agent combines diverse sandbox environments with a large-scale real-device mobile runtime. Its unified action space interleaves GUI operations with CLI execution and generates batched actions in a single model turn. An AutoResearch-style data flywheel uses agents to construct tasks and environments, diagnose failures, and plan subsequent iterations. Online RL supports training on trajectories exceeding 100 turns, with over 10,000 concurrent environments accelerating rollout. A lightweight harness layer supports proactive service initiation and stateful workflows across mobile and computer. Across a broad suite of evaluations, Qwen-UI-Agent sets state-of-the-art performance on mobile-use benchmarks while delivering competitive performance on computer- and browser-use tasks against frontier models, including Opus 4.8, Gemini 3.1 Pro, and GPT-5.6 Sol. On mobile use, it achieves 82.1% on MobileWorld, 92.2% on MobileWorld-Real, and 97.5% on AndroidDaily. On computer use, it achieves 79.5% on OSWorld-Verified and a 40.0% partial-progress score on OSWorld-v2. On browser use and GUI grounding, it achieves 73.6% on WebArena and 81.5% on ScreenSpot-Pro, respectively.