LDM3D-VR：用於3D虛擬實境的潛在擴散模型

摘要

潛在擴散模型已被證實在創建和操作視覺輸出方面處於領先地位。然而，就我們所知，深度地圖與 RGB 的生成仍然受到限制。我們引入了LDM3D-VR，這是一套針對虛擬實境開發的擴散模型，包括LDM3D-pano和LDM3D-SR。這些模型使得能夠基於文本提示生成全景RGBD，以及將低分辨率輸入升級為高分辨率RGBD。我們的模型是從包含全景/高分辨率RGB圖像、深度地圖和標題的數據集中微調而來的預訓練模型。這兩個模型與現有相關方法進行了評估比較。

English

Latent diffusion models have proven to be state-of-the-art in the creation and manipulation of visual outputs. However, as far as we know, the generation of depth maps jointly with RGB is still limited. We introduce LDM3D-VR, a suite of diffusion models targeting virtual reality development that includes LDM3D-pano and LDM3D-SR. These models enable the generation of panoramic RGBD based on textual prompts and the upscaling of low-resolution inputs to high-resolution RGBD, respectively. Our models are fine-tuned from existing pretrained models on datasets containing panoramic/high-resolution RGB images, depth maps and captions. Both models are evaluated in comparison to existing related methods.

LDM3D-VR：用於3D虛擬實境的潛在擴散模型

LDM3D-VR: Latent Diffusion Model for 3D VR

摘要

Support