Xiaomi-GUI-0 技術報告

摘要

圖形使用者介面（GUI）智能體以視覺語言模型為基礎，透過介面操作（如點擊、滑動、文字輸入與導航）在真實應用中端到端完成使用者任務。然而，現有的GUI智能體主要是在離線軌跡、模擬環境與標準化基準上進行訓練與評估。這些環境在介面佈局、互動邏輯與異常狀態分佈上與真實應用存在顯著差異，無法忠實反映實際使用情境下的執行穩定性——在真實環境中，帳號狀態、權限對話框、支付驗證與風險控制機制持續重塑狀態分佈，導致基準分數與實際可用性之間存在持久落差。為彌補此落差，我們提出Xiaomi-GUI-0，一個原生多模態GUI智能體，專為真實行動環境設計，並在實機閉環中完成訓練與評估。其核心在於一套以實機為主導的混合基礎設施：實體裝置作為主要執行環境，沙箱則提供輔助支援，使數據收集、訓練、推論與評估共享接近真實部署的執行分佈。我們構建多來源訓練數據，涵蓋高頻頭部任務、針對長尾意圖的高泛化數據，以及用於反思與記憶的能力增強數據，並引入錯誤驅動的數據飛輪，將失敗軌跡轉化為修正動作、反思說明與恢復示範。模型透過三階段漸進式訓練流程進行訓練：監督式微調、步驟級強化學習，以及智能體強化學習。在公開基準與自建的RealMobile上進行評估，Xiaomi-GUI-0在RealMobile上達到72.0%的成功率，在AndroidWorld上達到78.9%，同時在真實任務中顯著提升執行穩定性與異常狀態識別能力。

English

Graphical user interface (GUI) agents build on vision-language models to complete user tasks end-to-end in real applications through interface actions such as tapping, swiping, text entry, and navigation. However, existing GUI agents are trained and evaluated largely on offline trajectories, simulated environments, and standardized benchmarks. These differ substantially from real applications in interface layout, interaction logic, and abnormal-state distribution, and cannot faithfully characterize execution stability in real-world use, where account states, permission dialogs, payment authentication, and risk control continually reshape the state distribution and open a persistent gap between benchmark scores and real usability. To close this gap, we propose Xiaomi-GUI-0, a native multimodal GUI agent for real mobile environments, trained and evaluated within a real-device closed loop. At its core is a real-device-dominant hybrid infrastructure, where physical devices are the primary execution environment and sandboxes provide auxiliary support, so that data collection, training, rollout, and evaluation share an execution distribution close to real deployment. We construct multi-source training data spanning high-frequency head tasks, high-generalization data for long-tail intents, and capability-enhancement data for reflection and memory, and introduce an error-driven data flywheel that turns failure trajectories into corrected actions, reflective explanations, and recovery demonstrations. The model is trained through a progressive three-stage pipeline of supervised fine-tuning, step-level reinforcement learning, and agentic reinforcement learning. Evaluated on public benchmarks and our in-house RealMobile, Xiaomi-GUI-0 achieves 72.0% success on RealMobile and 78.9% on AndroidWorld, while substantially improving execution stability and abnormal-state recognition in real-world tasks.