SceneMosaic：透過混合代理式佈局演化實現高效且多樣之模擬就緒場景生成

摘要

多樣化且可直接模擬的室內場景對互動娛樂與具身人工智慧至關重要，然而其可擴展生成仍具挑戰性。近期依賴視覺語言模型（VLMs）的代理式文本到3D場景流程能生成高保真度的場景，但需要成本高昂的迭代物件放置與精煉。另一個主流範式，參數化圖像到3D場景模型，能從2D圖像中學習到的強先驗高效地生成場景，但往往導致不精確且物理上無效的場景。更重要的是，這兩種範式都難以針對單一輸入輸出多樣化的場景，使其難以反映真實場景的動態變化特性。在本文中，我們提出SceneMosaic，一個結合了兩種範式優點的框架。它從學習到的基於圖像的先驗中獲取初始候選，隨後透過VLM代理演化結果，確保效率與物理有效性。在演化過程中，SceneMosaic利用自然場景的局部性，將場景分解為獨立的局部單元，允許在每個單元內進行單獨演化，然後透過笛卡爾積組合全局場景。在SceneEval-100上，SceneMosaic在語義佈局品質方面媲美最強的代理式基線，並達到24倍加速，大幅減少物理違規，並獲得最高的人類評分。我們的程式碼公開於 https://github.com/rxjfighting/SceneMosaic。

English

Diverse and simulation-ready indoor scenes are essential for interactive entertainment and embodied AI, yet their scalable generation remains challenging. Recent agentic text-to-3D scene pipelines that rely on vision-language models (VLMs) can generate scenes of high fidelity but require costly iterative object placement and refinement. Another mainstream paradigm, parametric image-to-3D scene models, produces scenes efficiently from strong priors learned from 2D images but often leads to imprecise and physically invalid scenes. More importantly, both paradigms struggle to output diverse scenes for a single input, making it hard for them to reflect the dynamically changing nature of real scenes. In this paper we propose SceneMosaic, a framework that combines the merits of both paradigms. It obtains the initial candidate from the learned image-based prior, and subsequently evolves the result through VLM agents, ensuring both efficiency and physical validity. Within the evolution process, SceneMosaic exploits the locality of natural scenes and decomposes a scene into independent local units, allowing separate evolution within each unit before composing the global scene via Cartesian product. On SceneEval-100, SceneMosaic matches the strongest agentic baseline in semantic layout quality with a 24x speedup, substantially reduces physical violations, and receives the highest human ratings. Our code is publicly available at https://github.com/rxjfighting/SceneMosaic.