Lucida：用於可組合真實到模擬場景建模的解析、生成與放置

Lucida: Parse, Generate, and Place for Composable Real-to-Sim Scene Modeling

August 31, 2026

作者: Minghan Qin, Yuang Wang, Xiuyu Yang, Yushi Long, Yujian Zhang, Ruihuan Wang, Kai Ye, Yangang Zhang, Hang Li

cs.AI

摘要

可組合場景建模旨在將真實室內場景還原為完整的、可編輯的物體資產，並按照觀測到的排列方式呈現，為機器人模擬與具身AI提供一個可直接用於模擬的真實環境副本，其中物體可被逐一操控。現有管線將此任務分解為三個步驟——將觀測解析為實例、為每個實例生成資產、再將每個資產放回原位——但每個步驟都預設了一種雜亂場景捕捉難以提供的輸入：精確的實例幾何、無遮擋的視角，以及與觀測精確匹配的資產。我們提出Lucida，它保留了這一順序，但重新分配了各步驟的需求，使每個步驟僅使用真實捕捉能可靠提供的資訊，並在管線末端達成精確度，而非在起點就要求精確。Lucida將影片解析為場景圖，圖中節點承載每個實例的多視角證據，再從該證據為每個實例生成完整資產，並以GizmoAct放置資產。GizmoAct是一種視覺語言模型（VLM）策略，將放置任務轉化為多輪GUI互動，在閉環中操控物件的gizmo（操作控件），自行判斷何時達成對齊。在場景級3D物體偵測、物體姿態估計和場景重建方面，Lucida在R2S-Scene上相較Boxer將mAP提升了69%，在CA-1M上將ADD-SB@0.05從57.8%提高至83.4%，並將場景F-Score從SAM3D的0.794提升至0.924。

English

Composable scene modeling aims to recover a real indoor scene as complete, editable object assets arranged as observed, giving robot simulation and embodied AI a simulation-ready replica of the real environment whose objects can be manipulated individually. Existing pipelines decompose the task into three steps---parse the observations into instances, generate an asset for each, and place each asset back---but every step presumes an input that a cluttered capture rarely provides: accurate instance geometry, unoccluded views, and assets that accurately match the observations. We propose Lucida, which keeps this order but redistributes the requirements, so each step consumes only what a real capture reliably provides and precision is reached at the end of the pipeline rather than demanded at its start. Lucida parses the video into a scene graph whose nodes carry per-instance multi-view evidence, generates a complete asset for each instance from its evidence, and places assets with GizmoAct, a VLM policy that casts placement as multi-turn GUI interaction, manipulating the object's gizmo in a closed loop and deciding itself when alignment is reached. Across scene-level 3D object detection, object pose estimation, and scene reconstruction, Lucida improves mAP over Boxer by 69% on R2S-Scene, raises ADD-SB@0.05 from 57.8% to 83.4% on CA-1M, and increases scene F-Score from 0.794 for SAM3D to 0.924.