UI-MOPD: 面向持续GUI代理学习的多平台在策略蒸馏

摘要

近期，多模态基础模型与智能体系统的进展推动图形用户界面智能体从单平台任务执行向跨平台交互发展。然而，构建多平台GUI智能体仍面临挑战：一方面，高质量的、可执行的跨平台交互轨迹依然稀缺，现有数据往往存在平台覆盖范围有限的问题；另一方面，不同平台展现出差异化的交互规范，导致联合训练或持续训练容易引发行为模式混杂、特定平台能力退化以及灾难性遗忘。为解决这些问题，我们构建了高质量跨平台GUI交互数据集Uni-GUI，并提出了UI-MOPD方法——这是首个将多教师在线策略蒸馏融入GUI智能体持续学习的技术。UI-MOPD根据当前环境动态选择特定平台的教师模型，通过基于平台条件的蒸馏将平台特有的行为先验转移至共享策略，从而在保持现有平台能力的同时适应新平台。在OSWorld和MobileWorld上的实验表明，UI-MOPD的任务成功率分别达到38.2%和12.0%，验证了其在平衡跨平台能力保持与新平台适应方面的有效性。项目页面：https://elispectre.github.io/UI-MOPD/。

English

Recent advances in multimodal foundation models and agent systems have driven GUI agents from single-platform task execution toward cross-platform interaction. However, building multi-platform GUI agents remains challenging. On one hand, high-quality and executable cross-platform interaction trajectories are still scarce, and existing data often suffer from limited platform coverage. On the other hand, different platforms exhibit distinct interaction conventions, making joint or continual training prone to behavioral pattern mixing, platform-specific capability degradation, and catastrophic forgetting. To address these challenges, we construct Uni-GUI, a high-quality cross-platform GUI interaction dataset, and propose UI-MOPD, the first method that incorporates multi-teacher on-policy distillation into continual learning for GUI agents. UI-MOPD dynamically selects a platform-specific teacher according to the current environment and transfers platform-specific behavioral priors to a shared policy through platform-conditioned distillation, enabling adaptation to new platforms while preserving capabilities on existing ones. Experiments on OSWorld and MobileWorld show that UI-MOPD achieves task success rates of 38.2% and 12.0%, respectively, demonstrating its effectiveness in balancing cross-platform capability retention and new-platform adaptation. Project page: https://elispectre.github.io/UI-MOPD/.