影像雕塑：透過3D幾何控制進行精確物件編輯

Image Sculpting: Precise Object Editing with 3D Geometry Control

January 2, 2024

作者: Jiraphon Yenphraphai, Xichen Pan, Sainan Liu, Daniele Panozzo, Saining Xie

cs.AI

摘要

我們提出了圖像雕塑（Image Sculpting）這一新框架，通過融合3D幾何和圖形學工具，來編輯2D圖像。這種方法與現有方法有顯著不同，現有方法僅限於2D空間，通常依賴文字說明，導致模糊和受限控制。圖像雕塑將2D物體轉換為3D，使得可以直接與其3D幾何進行交互。在後期編輯中，這些物體重新渲染為2D，與原始圖像融合，通過粗到精的增強過程產生高保真度結果。該框架支持精確、可量化和物理合理的編輯選項，如姿勢編輯、旋轉、平移、3D合成、雕刻和串行添加。這標誌著將生成模型的創造自由與圖形管線的精確性相結合的初步步驟。

English

We present Image Sculpting, a new framework for editing 2D images by incorporating tools from 3D geometry and graphics. This approach differs markedly from existing methods, which are confined to 2D spaces and typically rely on textual instructions, leading to ambiguity and limited control. Image Sculpting converts 2D objects into 3D, enabling direct interaction with their 3D geometry. Post-editing, these objects are re-rendered into 2D, merging into the original image to produce high-fidelity results through a coarse-to-fine enhancement process. The framework supports precise, quantifiable, and physically-plausible editing options such as pose editing, rotation, translation, 3D composition, carving, and serial addition. It marks an initial step towards combining the creative freedom of generative models with the precision of graphics pipelines.