ChatPaper.aiChatPaper

图像雕塑:利用3D几何控制进行精确对象编辑

Image Sculpting: Precise Object Editing with 3D Geometry Control

January 2, 2024
作者: Jiraphon Yenphraphai, Xichen Pan, Sainan Liu, Daniele Panozzo, Saining Xie
cs.AI

摘要

我们提出了图像雕塑,这是一个通过整合来自3D几何和图形学工具来编辑2D图像的新框架。这种方法与现有方法有显著区别,现有方法局限于2D空间,通常依赖文本指令,导致歧义和受限控制。图像雕塑将2D对象转换为3D,使得可以直接与它们的3D几何进行交互。在后期编辑中,这些对象被重新渲染为2D,与原始图像融合,通过粗到精的增强过程产生高保真的结果。该框架支持精确、可量化和物理合理的编辑选项,如姿势编辑、旋转、平移、3D组合、雕刻和串行添加。它标志着将生成模型的创造自由与图形管线的精度结合的初步步骤。
English
We present Image Sculpting, a new framework for editing 2D images by incorporating tools from 3D geometry and graphics. This approach differs markedly from existing methods, which are confined to 2D spaces and typically rely on textual instructions, leading to ambiguity and limited control. Image Sculpting converts 2D objects into 3D, enabling direct interaction with their 3D geometry. Post-editing, these objects are re-rendered into 2D, merging into the original image to produce high-fidelity results through a coarse-to-fine enhancement process. The framework supports precise, quantifiable, and physically-plausible editing options such as pose editing, rotation, translation, 3D composition, carving, and serial addition. It marks an initial step towards combining the creative freedom of generative models with the precision of graphics pipelines.
PDF211December 15, 2024