訓練開放模型進行自主手機操作

摘要

手機正逐漸成為通用型智能體的重要執行平台，但訓練可靠操作手機的開放模型仍具挑戰，因為部署時真正重要的環境——真實裝置上運行的真實應用——速度慢、具狀態、有副作用，且難以重置或驗證，而可擴展的模擬環境僅能近似真實行為。我們提出 PhoneBuddy：一套專為手機智能體操作設計的訓練配方與開放模型系列，其結合了真實應用環境與模擬應用環境 PhoneWorld——後者可從真實 GUI 使用結構中還原出可運行的模擬應用。PhoneBuddy 首先透過在兩種環境中收集的軌跡進行共享的監督式微調階段，接著在純真實應用強化學習與混合兩種環境的強化學習之間進行比較。在一項涵蓋 150 項任務、於真實手機上進行的人類評估中（任務橫跨一般應用、迷你應用及跨應用工作流程），任務成功率從監督式微調後的 36.67% 提升至純真實應用強化學習後的 40.67%，以及混合強化學習後的 45.33%。在 AndroidWorld 上，同樣的進程從 60.3% 上升至 77.2%，再至 83.2%。這些結果顯示，模擬應用訓練並非取代真實應用強化學習，而是提供可擴展、可重置且可自動檢查互動的一種互補來源。其增益在一般應用與迷你應用任務上最為明顯，而長時間跨應用工作流程仍是一項重要的開放性挑戰。

English

Phones are becoming an important execution surface for general-purpose agents, but training open models for reliable phone use remains difficult because the environment that matters at deployment, real devices running real apps, is slow, stateful, side-effectful, and hard to reset or verify, while scalable mock environments only approximate real behavior. We present PhoneBuddy, a training recipe and open-model line for agentic phone use that combines a real-app environment with a mock-app environment, PhoneWorld, which reconstructs runnable mock apps from real GUI usage structure. PhoneBuddy first builds a shared supervised fine-tuning stage from trajectories collected in both environments, then compares real-app RL against mixed RL across both environments. Across a 150-task human evaluation on real phones spanning apps, mini-apps, and cross-app workflows, task success rate improves from 36.67\% after supervised fine-tuning to 40.67\% after real-app RL and 45.33\% after mixed RL. On AndroidWorld, the same progression rises from 60.3\% to 77.2\% to 83.2\%. These results show that mock-app training is not a replacement for real-app RL, but a complementary source of scalable, resettable, and automatically checked interaction. The gains are strongest on app and mini-app tasks, while long-horizontal cross-app workflows remain an important open challenge.