CAT3D：使用多視圖擴散模型在3D中創建任何物件

CAT3D: Create Anything in 3D with Multi-View Diffusion Models

May 16, 2024

作者: Ruiqi Gao, Aleksander Holynski, Philipp Henzler, Arthur Brussee, Ricardo Martin-Brualla, Pratul Srinivasan, Jonathan T. Barron, Ben Poole

cs.AI

摘要

3D重建技術的進步已經實現高質量的3D捕捉，但需要用戶收集數百至數千張圖像來創建3D場景。我們提出了CAT3D，一種通過模擬這種現實世界捕捉過程的多視圖擴散模型來創建3D中的任何物體的方法。給定任意數量的輸入圖像和一組目標新視角，我們的模型生成高度一致的場景新視角。這些生成的視角可以作為強大的3D重建技術的輸入，以產生可以從任何視角實時渲染的3D表示。CAT3D可以在一分鐘內創建完整的3D場景，並且優於現有的單圖像和少視角3D場景創建方法。請查看我們的項目頁面以獲得結果和互動演示：https://cat3d.github.io。

English

Advances in 3D reconstruction have enabled high-quality 3D capture, but require a user to collect hundreds to thousands of images to create a 3D scene. We present CAT3D, a method for creating anything in 3D by simulating this real-world capture process with a multi-view diffusion model. Given any number of input images and a set of target novel viewpoints, our model generates highly consistent novel views of a scene. These generated views can be used as input to robust 3D reconstruction techniques to produce 3D representations that can be rendered from any viewpoint in real-time. CAT3D can create entire 3D scenes in as little as one minute, and outperforms existing methods for single image and few-view 3D scene creation. See our project page for results and interactive demos at https://cat3d.github.io .