GroundingPI：一個以視覺基元邁向物理智能的基礎模型

摘要

精確定位至關重要。它指明了目標物為何、以及該物件位於何處，即使在雜亂場景中或面對微小物體亦然，且速度必須足夠快以支援閉環控制。然而，視覺-語言-動作（VLA）模型與世界-動作模型（WAMs）從通用視覺-語言與影片生成的骨幹網路中獲取感知能力，而這些骨幹在上述情境中仍會失敗。我們提出 GroundingPI，一個 4B 參數的定位基礎模型，以共享詞彙表中的量化座標生成點與框。其訓練結合了多模態與空間預訓練、監督式微調，以及採用 GRPO 的強化學習，並使用來自公開資料集與專用資料引擎的監督訊號。在涵蓋 11 種感知能力、34 項定位基準的 44 個基線方法中，GroundingPI 樹立了新的技術水準，平均達 73.68%，超越規模更大的 GPT-6 Astra（71.54%）。作為下游視覺骨幹，GroundingPI 提升了機器人操作與自動駕駛的效能。在 RoboTwin 2.0 上，它在我們評估的全部四種分布外設定中均優於所有主流骨幹，相較於最強骨幹最高提升 24.8%。在 RoboCasa-GR1 上，僅以 50% 示範資料訓練的 GroundingPI 即優於以 75% 資料訓練的基線方法。在 nuScenes 上，作為視覺骨幹使用時，GroundingPI 達到平均開環 L2 誤差 0.296 公尺。我們系統性地分析了 GroundingPI 預訓練的規模與資料組成。隨著預訓練規模擴大，下游的自動駕駛與機器人操作效能隨之提升。針對這 11 種感知能力分析資料配方顯示，密集定位對兩者皆帶來顯著效益，而 OCR 具有作為感知學習催化劑的潛力。這些結果支持了定位作為感知基礎的地位，以及專用感知預訓練作為物理智慧基礎模型的一條具前景之方向。

English

Precise grounding matters. It specifies which object is the target and where that object is, even in clutter and for tiny objects, and it has to be fast enough for closed-loop control. Yet vision-language-action (VLA) and world-action models (WAMs) take perception from general-purpose vision-language and video-generation backbones, which still fail in these settings. We introduce GroundingPI, a 4B grounding foundation model that generates points and boxes as quantized coordinates in a shared vocabulary. Training combines multimodal and spatial pretraining, supervised fine-tuning, and reinforcement learning with GRPO, using supervision from public datasets and dedicated data engines. Against 44 baselines across 34 grounding benchmarks spanning 11 perceptual capabilities, GroundingPI establishes a new state of the art, averaging 73.68%, above the larger GPT-6 Astra (71.54%). As a downstream visual backbone, GroundingPI improves performance on robotic manipulation and autonomous driving. On RoboTwin 2.0, it outperforms every mainstream backbone we evaluate in all four out-of-distribution settings, by up to 24.8% relative to the strongest backbone. On RoboCasa-GR1, GroundingPI trained with 50% of the demonstrations outperforms those baselines trained with 75%. On nuScenes, used as the visual backbone, GroundingPI attains an average open-loop L2 error of 0.296 m. We systematically analyze GroundingPI's pretraining in scale and data composition. Downstream autonomous driving and robotic manipulation improve as the pretraining is scaled. Analyzing the data recipe across these 11 perceptual capabilities shows dense grounding's substantial benefits for both, and OCR's potential as a catalyst for perceptual learning. These results support grounding as a perceptual foundation, and dedicated perceptual pretraining as a promising direction for foundation models of physical intelligence.

GroundingPI：一個以視覺基元邁向物理智能的基礎模型

GroundingPI: A Grounding Foundation Model towards Physical Intelligence with Visual Primitives

摘要

Support