一次前向勝過兩次：InnerZoom 實現精準高效的 GUI 基礎對應

摘要

基於MLLM的圖形用戶界面定位方法通常將目標定位形式化為自回歸坐標生成，使模型能夠利用MLLM強大的指令遵循與語義理解能力。然而，這種形式要求模型在解碼坐標標記的同時，保留區域級目標證據，以滿足GUI點擊所需的空間精度。我們的診斷分析顯示，目標區域感知能力在解碼器中間層湧現，但既未被保留，也未被轉化為最終的坐標預測。現有的ZoomIn式方法透過外部裁剪重跑流程解決此問題，雖可提升定位精度，卻增加了端到端延遲與計算成本。為在無額外成本下保留兩次縮放定位的精度優勢，我們提出InnerZoom——一種用於跨層證據橋接的單次前向框架。InnerZoom將原始前向傳遞中的目標相關線索轉換為緊湊的跨層證據狀態，隨後在後續解碼層中保留、精煉並重新注入該狀態，以引導坐標預測。大量實驗結果表明，InnerZoom-4B在所有六項GUI定位基準上均達到了最先進性能：OSWorld-G取得64.7分，UI-Vision取得40.2分，OSWorld-GR取得73.1分，MMBench-GUI取得87.6分，分別超越先前最佳成績4.1、3.2、2.9與2.3分。在可控的4B設定下，InnerZoom將同等的SFT+RL基線平均提升5.3分，並平均優於兩次縮放ZoomIn方法1.3分，同時端到端延遲降低最多31.8%，TFLOPs減少約29%。程式碼與模型將公開提供。

English

MLLM-based GUI grounding methods commonly formulate target localization as autoregressive coordinate generation, enabling models to leverage the strong instruction-following and semantic understanding capabilities of MLLMs. However, this formulation requires the model to retain region-level target evidence while decoding coordinate tokens with the spatial precision demanded by GUI clicking. Our diagnostic analysis reveals that target-region awareness emerges in intermediate decoder layers but is neither retained nor translated into the final coordinate prediction. Existing ZoomIn-style methods address this issue through an external crop-and-rerun pass, which improves localization but increases end-to-end latency and computational cost. To retain the accuracy benefits of two-pass zooming without this extra cost, we propose InnerZoom, a single-forward framework for cross-layer evidence bridging. InnerZoom transforms target-related cues from the original forward pass into a compact cross-layer evidence state, then preserves, refines, and reinjects this state throughout later decoding layers to guide coordinate prediction. Extensive experimental results suggest that InnerZoom-4B achieves state-of-the-art performance on all six GUI grounding benchmarks, obtaining 64.7 on OSWorld-G, 40.2 on UI-Vision, 73.1 on OSWorld-GR, and 87.6 on MMBench-GUI, surpassing the previous best results by 4.1, 3.2, 2.9, and 2.3 points, respectively. Under a controlled 4B setting, InnerZoom improves the same SFT+RL baseline by 5.3 points on average and outperforms two-pass ZoomIn by 1.3 points on average, while reducing end-to-end latency by up to 31.8% and TFLOPs by about 29%. Code and models will be publicly available.