GroundingPI：一个面向物理智能、以视觉基元为支撑的接地基础模型

摘要

精准定位至关重要。它指明哪个物体是目标以及该物体位于何处，即使在杂乱场景和微小物体中也是如此，并且必须足够快以支持闭环控制。然而，视觉-语言-动作（VLA）模型和世界-动作模型（WAMs）从通用视觉-语言和视频生成骨干中获取感知能力，而这些骨干在这些场景中仍然表现不佳。我们提出 GroundingPI，一个 4B 定位基础模型，它以共享词表中的量化坐标生成点和框。训练结合了多模态与空间预训练、有监督微调以及使用 GRPO 的强化学习，并利用来自公开数据集和专用数据引擎的监督。在涵盖 11 种感知能力的 34 个定位基准上，与 44 个基线相比，GroundingPI 树立了新的当前最优，平均达到 73.68%，高于规模更大的 GPT-6 Astra（71.54%）。作为下游视觉骨干，GroundingPI 提升了机器人操作和自动驾驶的性能。在 RoboTwin 2.0 上，它在所有四种分布外设置下都优于我们评估的每一个主流骨干，相对最强骨干最高提升 24.8%。在 RoboCasa-GR1 上，仅用 50% 演示训练的 GroundingPI 超过了用 75% 演示训练的基线。在 nuScenes 上，作为视觉骨干使用时，GroundingPI 达到 0.296 米的平均开环 L2 误差。我们系统分析了 GroundingPI 预训练的规模和数据组成。随着预训练规模扩大，下游自动驾驶和机器人操作性能得到提升。对这些 11 种感知能力的数据配方进行分析表明，密集定位对二者都有显著益处，而 OCR 具有作为感知学习催化剂的潜力。这些结果支持将定位作为感知基础，也支持将专门的感知预训练作为物理智能基础模型的一个有前景的方向。

English

Precise grounding matters. It specifies which object is the target and where that object is, even in clutter and for tiny objects, and it has to be fast enough for closed-loop control. Yet vision-language-action (VLA) and world-action models (WAMs) take perception from general-purpose vision-language and video-generation backbones, which still fail in these settings. We introduce GroundingPI, a 4B grounding foundation model that generates points and boxes as quantized coordinates in a shared vocabulary. Training combines multimodal and spatial pretraining, supervised fine-tuning, and reinforcement learning with GRPO, using supervision from public datasets and dedicated data engines. Against 44 baselines across 34 grounding benchmarks spanning 11 perceptual capabilities, GroundingPI establishes a new state of the art, averaging 73.68%, above the larger GPT-6 Astra (71.54%). As a downstream visual backbone, GroundingPI improves performance on robotic manipulation and autonomous driving. On RoboTwin 2.0, it outperforms every mainstream backbone we evaluate in all four out-of-distribution settings, by up to 24.8% relative to the strongest backbone. On RoboCasa-GR1, GroundingPI trained with 50% of the demonstrations outperforms those baselines trained with 75%. On nuScenes, used as the visual backbone, GroundingPI attains an average open-loop L2 error of 0.296 m. We systematically analyze GroundingPI's pretraining in scale and data composition. Downstream autonomous driving and robotic manipulation improve as the pretraining is scaled. Analyzing the data recipe across these 11 perceptual capabilities shows dense grounding's substantial benefits for both, and OCR's potential as a catalyst for perceptual learning. These results support grounding as a perceptual foundation, and dedicated perceptual pretraining as a promising direction for foundation models of physical intelligence.

GroundingPI：一个面向物理智能、以视觉基元为支撑的接地基础模型

GroundingPI: A Grounding Foundation Model towards Physical Intelligence with Visual Primitives

摘要

Support