GeneralVLA-2：幾何感知重建與受控記憶用於機器人規劃

摘要

通用视觉-语言-动作系統需要以物體為中心的3D證據和可重複使用的操作經驗，才能規劃可靠的機器人軌跡。GeneralVLA提供了一個分層介面，可將語言和RGB-D觀測轉換為3D末端執行器路徑，但仍存在兩個瓶頸。首先，單目SAM3D風格的物體重建可能產生姿態和未見幾何結構的幻覺，而當有校正後的多視角觀測資料可用時，操作任務則受益於穩定的物體形狀。其次，原始的KnowledgeBank主要檢索語義相似的片段並追加新知識，這使得難以控制記憶品質、衝突、置信度以及幾何相關性。為解決第一個挑戰，我們引入GeoFuse-MV3D，一種基於幾何先驗引導的MV-SAM3D重建分支，它利用輸入視角遮罩驗證外部幾何線索，應用軟視覺外殼支持，進行逐軸精細化，並僅融合幾何資訊而保留外觀。為解決第二個挑戰，我們將KnowledgeBank升級為一個受控的長期記憶系統，配備明確的品質、置信度、生命週期、驗證器和衝突元數據，以及精度導向的檢索機制。最後，我們在GSO-30上評估重建分支，在Terminal-Bench 2.0和SWE-Bench Verified上評估記憶模組；GeoFuse-MV3D相比MV-SAM3D基線，CD和LPIPS分別降低2.20%和2.02%，PSNR和SSIM分別提升2.36%和1.03%；KnowledgeBank在Terminal-Bench SR上相比ReasoningBank提升4.53%，在SWE-Bench解決率上提升3.73%，同時AS分別降低4.95%和5.65%。程式碼：https://github.com/AIGeeksGroup/GeneralVLA-2。網站：https://aigeeksgroup.github.io/GeneralVLA-2。

English

Generalist vision-language-action systems need object-centric 3D evidence and reusable manipulation experience to plan reliable robot trajectories. GeneralVLA provides a hierarchical interface for converting language and RGB-D observations into 3D end-effector paths, but two bottlenecks remain. First, monocular SAM3D-style object reconstruction can hallucinate pose and unseen geometry, while manipulation benefits from stable object shape when calibrated multi-view observations are available. Second, the original KnowledgeBank mainly retrieves semantically similar snippets and appends new knowledge, which makes it difficult to control memory quality, conflicts, confidence, and geometric relevance. To address the first challenge, we introduce GeoFuse-MV3D, a geometry-prior-guided MV-SAM3D reconstruction branch that verifies external geometry cues with input-view masks, applies soft visual-hull support, performs axis-wise refinement, and fuses only geometry while preserving appearance. To address the second challenge, we upgrade KnowledgeBank into a governed long-term memory system with explicit quality, confidence, lifecycle, verifier, and conflict metadata, together with precision-oriented retrieval. Finally, we evaluate the reconstruction branch on GSO-30 and the memory module on Terminal-Bench 2.0 and SWE-Bench Verified; GeoFuse-MV3D improves over the MV-SAM3D baseline by reducing CD and LPIPS by 2.20% and 2.02% while increasing PSNR and SSIM by 2.36% and 1.03%, and KnowledgeBank improves over ReasoningBank by 4.53% on Terminal-Bench SR and 3.73% on SWE-Bench resolve rate, while reducing AS by 4.95% and 5.65%, respectively. Code: https://github.com/AIGeeksGroup/GeneralVLA-2. Website: https://aigeeksgroup.github.io/GeneralVLA-2.