MagicClay：使用生成神經場塑造網格

摘要

最近神經領域的發展為形狀生成領域帶來了非凡的能力，但它們缺乏關鍵特性，例如增量控制 - 這是藝術工作的基本要求。另一方面，三角網格是大多數幾何相關任務的首選表示形式，提供效率和直觀控制，但不適合進行神經優化。為了支持下游任務，先前的藝術作品通常提出了一種兩步方法，首先使用神經領域生成形狀，然後提取網格進行進一步處理。相反，在本文中，我們介紹了一種混合方法，可以始終保持網格和符號距離場（SDF）表示的一致性。利用這種表示，我們引入了MagicClay - 一個友好的藝術家工具，可以根據文本提示雕塑網格的區域，同時保持其他區域不變。我們的框架在每個形狀優化步驟中仔細而有效地平衡表示之間的一致性和正則化；依賴網格表示，我們展示了如何以更高的分辨率和更快的速度呈現SDF。此外，我們利用最近的可微網格重建工作，根據SDF指示，自適應地分配網格中所需的三角形。通過實現的原型，我們展示了與最先進技術相比更優秀的生成幾何形狀，以及新穎的一致控制，首次允許對同一網格進行基於順序提示的編輯。

English

The recent developments in neural fields have brought phenomenal capabilities to the field of shape generation, but they lack crucial properties, such as incremental control - a fundamental requirement for artistic work. Triangular meshes, on the other hand, are the representation of choice for most geometry related tasks, offering efficiency and intuitive control, but do not lend themselves to neural optimization. To support downstream tasks, previous art typically proposes a two-step approach, where first a shape is generated using neural fields, and then a mesh is extracted for further processing. Instead, in this paper we introduce a hybrid approach that maintains both a mesh and a Signed Distance Field (SDF) representations consistently. Using this representation, we introduce MagicClay - an artist friendly tool for sculpting regions of a mesh according to textual prompts while keeping other regions untouched. Our framework carefully and efficiently balances consistency between the representations and regularizations in every step of the shape optimization; Relying on the mesh representation, we show how to render the SDF at higher resolutions and faster. In addition, we employ recent work in differentiable mesh reconstruction to adaptively allocate triangles in the mesh where required, as indicated by the SDF. Using an implemented prototype, we demonstrate superior generated geometry compared to the state-of-the-art, and novel consistent control, allowing sequential prompt-based edits to the same mesh for the first time.

MagicClay：使用生成神經場塑造網格

MagicClay: Sculpting Meshes With Generative Neural Fields

摘要

Support