StyleSplat：使用高斯点喷涂进行3D物体风格转移

StyleSplat: 3D Object Style Transfer with Gaussian Splatting

July 12, 2024

作者: Sahil Jain, Avik Kuthiala, Prabhdeep Singh Sethi, Prakanshul Saxena

cs.AI

摘要

最近在光辐场方面的进展为创建高质量的3D资产和场景开辟了新途径。风格迁移可以利用不同的艺术风格增强这些3D资产，从而转变创意表达。然而，现有技术通常速度较慢，或无法将风格迁移局限于特定对象。我们引入了StyleSplat，一种轻量级方法，用于通过来自参考风格图像的3D高斯函数对场景中的3D对象进行样式化。我们的方法首先使用3D高斯函数喷洒学习场景的照片级表示，同时分割单个3D对象。然后，我们使用最近邻特征匹配损失来微调所选对象的高斯函数，将它们的球谐系数与风格图像对齐，以确保一致性和视觉吸引力。StyleSplat允许快速、可定制的风格迁移，并在场景中局部实现多个对象的样式化，每个对象具有不同的风格。我们展示了它在各种3D场景和风格中的有效性，展示了在3D创作中增强的控制和定制能力。

English

Recent advancements in radiance fields have opened new avenues for creating high-quality 3D assets and scenes. Style transfer can enhance these 3D assets with diverse artistic styles, transforming creative expression. However, existing techniques are often slow or unable to localize style transfer to specific objects. We introduce StyleSplat, a lightweight method for stylizing 3D objects in scenes represented by 3D Gaussians from reference style images. Our approach first learns a photorealistic representation of the scene using 3D Gaussian splatting while jointly segmenting individual 3D objects. We then use a nearest-neighbor feature matching loss to finetune the Gaussians of the selected objects, aligning their spherical harmonic coefficients with the style image to ensure consistency and visual appeal. StyleSplat allows for quick, customizable style transfer and localized stylization of multiple objects within a scene, each with a different style. We demonstrate its effectiveness across various 3D scenes and styles, showcasing enhanced control and customization in 3D creation.