Lucida：面向可组合真实到仿真场景建模的解析、生成与放置

摘要

可组合场景建模旨在将真实室内场景恢复为按观察排列的完整、可编辑对象资产，从而为机器人仿真和具身人工智能提供真实环境的可直接用于仿真的副本，其中每个对象均可独立操控。现有流程将该任务分解为三个步骤——将观测解析为实例、为每个实例生成资产、并将每个资产放回原位——但每一步都预设了杂乱采集难以提供的输入：精确的实例几何、无遮挡的视角以及与观测精确匹配的资产。我们提出Lucida，它保持这一顺序但重新分配需求，使得每一步仅消耗真实采集可靠提供的信息，并且精度在流程末端达成，而非在起点就被要求。Lucida将视频解析为一个场景图，其节点携带每个实例的多视角证据；然后从这些证据为每个实例生成完整资产；并使用GizmoAct放置资产。GizmoAct是一种视觉语言模型（VLM）策略，它将放置任务转化为多轮GUI交互，在闭环中操控对象的gizmo（变换控件），并自行判断何时达到对齐。在场景级3D目标检测、对象姿态估计和场景重建中，Lucida在R2S-Scene上将mAP相比Boxer提升了69%，在CA-1M上将ADD-SB@0.05从57.8%提升至83.4%，并将场景F-Score从SAM3D的0.794提升至0.924。

English

Composable scene modeling aims to recover a real indoor scene as complete, editable object assets arranged as observed, giving robot simulation and embodied AI a simulation-ready replica of the real environment whose objects can be manipulated individually. Existing pipelines decompose the task into three steps---parse the observations into instances, generate an asset for each, and place each asset back---but every step presumes an input that a cluttered capture rarely provides: accurate instance geometry, unoccluded views, and assets that accurately match the observations. We propose Lucida, which keeps this order but redistributes the requirements, so each step consumes only what a real capture reliably provides and precision is reached at the end of the pipeline rather than demanded at its start. Lucida parses the video into a scene graph whose nodes carry per-instance multi-view evidence, generates a complete asset for each instance from its evidence, and places assets with GizmoAct, a VLM policy that casts placement as multi-turn GUI interaction, manipulating the object's gizmo in a closed loop and deciding itself when alignment is reached. Across scene-level 3D object detection, object pose estimation, and scene reconstruction, Lucida improves mAP over Boxer by 69% on R2S-Scene, raises ADD-SB@0.05 from 57.8% to 83.4% on CA-1M, and increases scene F-Score from 0.794 for SAM3D to 0.924.

Lucida：面向可组合真实到仿真场景建模的解析、生成与放置

Lucida: Parse, Generate, and Place for Composable Real-to-Sim Scene Modeling

摘要

Support