CAT3D：使用多视角扩散模型在3D中创作任何内容

CAT3D: Create Anything in 3D with Multi-View Diffusion Models

May 16, 2024

作者: Ruiqi Gao, Aleksander Holynski, Philipp Henzler, Arthur Brussee, Ricardo Martin-Brualla, Pratul Srinivasan, Jonathan T. Barron, Ben Poole

cs.AI

摘要

3D重建的进展实现了高质量的3D捕捉，但需要用户收集数百到数千张图像来创建一个3D场景。我们提出了CAT3D，一种通过模拟这个真实世界捕捉过程的多视角扩散模型来创建任何3D物体的方法。给定任意数量的输入图像和一组目标新视点，我们的模型生成高度一致的场景新视图。这些生成的视图可用作强大的3D重建技术的输入，以生成可以实时从任何视点渲染的3D表示。CAT3D可以在一分钟内创建完整的3D场景，并且在单图像和少视角3D场景创建方面优于现有方法。请访问我们的项目页面以查看结果和交互式演示：https://cat3d.github.io。

English

Advances in 3D reconstruction have enabled high-quality 3D capture, but require a user to collect hundreds to thousands of images to create a 3D scene. We present CAT3D, a method for creating anything in 3D by simulating this real-world capture process with a multi-view diffusion model. Given any number of input images and a set of target novel viewpoints, our model generates highly consistent novel views of a scene. These generated views can be used as input to robust 3D reconstruction techniques to produce 3D representations that can be rendered from any viewpoint in real-time. CAT3D can create entire 3D scenes in as little as one minute, and outperforms existing methods for single image and few-view 3D scene creation. See our project page for results and interactive demos at https://cat3d.github.io .