StyleSplat：使用高斯點陣實現的3D物體風格轉移

StyleSplat: 3D Object Style Transfer with Gaussian Splatting

July 12, 2024

作者: Sahil Jain, Avik Kuthiala, Prabhdeep Singh Sethi, Prakanshul Saxena

cs.AI

摘要

最近對輝度場的進展為創建高質量的3D資產和場景開辟了新的途徑。風格轉移可以通過不同的藝術風格增強這些3D資產，轉換創意表達。然而，現有技術通常速度較慢，或無法將風格轉移局部化到特定物體。我們介紹了StyleSplat，這是一種輕量級的方法，用於對由參考風格圖像中的3D高斯表示的場景中的3D物體進行風格化。我們的方法首先通過3D高斯飛灰技術學習場景的照片逼真表示，同時分割單個3D物體。然後，我們使用最近鄰特徵匹配損失來微調所選物體的高斯函數，將它們的球面諧波係數與風格圖像對齊，以確保一致性和視覺吸引力。StyleSplat允許快速、可定制的風格轉移和局部風格化多個物體在一個場景中，每個物體都具有不同的風格。我們展示了它在各種3D場景和風格中的有效性，展示了在3D創作中增強的控制和定制能力。

English

Recent advancements in radiance fields have opened new avenues for creating high-quality 3D assets and scenes. Style transfer can enhance these 3D assets with diverse artistic styles, transforming creative expression. However, existing techniques are often slow or unable to localize style transfer to specific objects. We introduce StyleSplat, a lightweight method for stylizing 3D objects in scenes represented by 3D Gaussians from reference style images. Our approach first learns a photorealistic representation of the scene using 3D Gaussian splatting while jointly segmenting individual 3D objects. We then use a nearest-neighbor feature matching loss to finetune the Gaussians of the selected objects, aligning their spherical harmonic coefficients with the style image to ensure consistency and visual appeal. StyleSplat allows for quick, customizable style transfer and localized stylization of multiple objects within a scene, each with a different style. We demonstrate its effectiveness across various 3D scenes and styles, showcasing enhanced control and customization in 3D creation.