Repaint123：快速高质量的一图到三维生成，具有渐进可控的二维重绘

摘要

最近，一种常见的将单张图像转换为3D的方法采用了得分蒸馏采样（SDS）。尽管取得了令人印象深刻的结果，但存在多个缺陷，包括多视角不一致、过度饱和和过度平滑的纹理，以及生成速度慢。为了解决这些问题，我们提出了Repaint123，以减轻多视角偏差以及纹理退化，并加快生成过程。其核心思想是结合2D扩散模型的强大图像生成能力和修复策略的纹理对齐能力，生成具有一致性的高质量多视角图像。我们进一步提出了适用于重叠区域的可见性感知自适应修复强度，以增强修复过程中生成图像的质量。生成的高质量和多视角一致的图像使得可以使用简单的均方误差（MSE）损失进行快速3D内容生成。我们进行了大量实验，并展示了我们的方法能够在2分钟内从头开始生成具有高质量、多视角一致性和精细纹理的3D内容的优越能力。代码位于https://github.com/junwuzhang19/repaint123。

English

Recent one image to 3D generation methods commonly adopt Score Distillation Sampling (SDS). Despite the impressive results, there are multiple deficiencies including multi-view inconsistency, over-saturated and over-smoothed textures, as well as the slow generation speed. To address these deficiencies, we present Repaint123 to alleviate multi-view bias as well as texture degradation and speed up the generation process. The core idea is to combine the powerful image generation capability of the 2D diffusion model and the texture alignment ability of the repainting strategy for generating high-quality multi-view images with consistency. We further propose visibility-aware adaptive repainting strength for overlap regions to enhance the generated image quality in the repainting process. The generated high-quality and multi-view consistent images enable the use of simple Mean Square Error (MSE) loss for fast 3D content generation. We conduct extensive experiments and show that our method has a superior ability to generate high-quality 3D content with multi-view consistency and fine textures in 2 minutes from scratch. Code is at https://github.com/junwuzhang19/repaint123.

Repaint123：快速高质量的一图到三维生成，具有渐进可控的二维重绘

Repaint123: Fast and High-quality One Image to 3D Generation with Progressive Controllable 2D Repainting

摘要

Support