MagicClay：利用生成神经场塑造网格

摘要

最近神经场领域的发展为形状生成领域带来了非凡的能力，但它们缺乏关键属性，比如增量控制 - 这是艺术工作的基本要求。另一方面，三角网格是大多数几何相关任务的首选表示，提供了高效和直观的控制，但不适合神经优化。为了支持下游任务，先前的研究通常提出了一个两步方法，首先使用神经场生成形状，然后提取网格以进行进一步处理。相反，在本文中，我们介绍了一种混合方法，保持网格和有符号距离场（SDF）表示的一致性。利用这种表示，我们引入了MagicClay - 一种艺术家友好的工具，可以根据文本提示对网格的区域进行雕塑，同时保持其他区域不变。我们的框架在每一步的形状优化中谨慎而高效地平衡了表示和正则化之间的一致性；依靠网格表示，我们展示了如何以更高的分辨率和更快的速度渲染SDF。此外，我们利用最近的可微分网格重建工作，根据SDF指示，自适应地分配网格中所需的三角形。通过一个实现的原型，我们展示了与最先进技术相比更优秀的生成几何形状，并且具有新颖的一致控制，首次允许基于顺序提示对同一网格进行编辑。

English

The recent developments in neural fields have brought phenomenal capabilities to the field of shape generation, but they lack crucial properties, such as incremental control - a fundamental requirement for artistic work. Triangular meshes, on the other hand, are the representation of choice for most geometry related tasks, offering efficiency and intuitive control, but do not lend themselves to neural optimization. To support downstream tasks, previous art typically proposes a two-step approach, where first a shape is generated using neural fields, and then a mesh is extracted for further processing. Instead, in this paper we introduce a hybrid approach that maintains both a mesh and a Signed Distance Field (SDF) representations consistently. Using this representation, we introduce MagicClay - an artist friendly tool for sculpting regions of a mesh according to textual prompts while keeping other regions untouched. Our framework carefully and efficiently balances consistency between the representations and regularizations in every step of the shape optimization; Relying on the mesh representation, we show how to render the SDF at higher resolutions and faster. In addition, we employ recent work in differentiable mesh reconstruction to adaptively allocate triangles in the mesh where required, as indicated by the SDF. Using an implemented prototype, we demonstrate superior generated geometry compared to the state-of-the-art, and novel consistent control, allowing sequential prompt-based edits to the same mesh for the first time.

MagicClay：利用生成神经场塑造网格

MagicClay: Sculpting Meshes With Generative Neural Fields

摘要

Support