利用混合部分卷积改進遮罩風格轉換

Improving Masked Style Transfer using Blended Partial Convolution

August 7, 2025

作者: Seyed Hadi Seyed, Ayberk Cansever, David Hart

cs.AI

摘要

隨著卷積神經網絡和基於Transformer的神經網絡的發展，藝術風格遷移技術早已成為可能。大多數算法將藝術風格遷移應用於整幅圖像，但個別用戶可能僅需對圖像中的特定區域進行風格遷移。標準做法是在風格化後簡單地對圖像進行遮罩處理。本研究表明，這種方法往往無法正確捕捉感興趣區域的風格特徵。我們提出了一種基於部分卷積的風格遷移網絡，能夠精確地將風格特徵僅應用於感興趣區域。此外，我們還提出了網絡內部融合技術，以應對區域選擇中的不完美之處。我們通過SA-1B數據集中的示例展示，這種方法在視覺和量化上都提升了風格化效果。代碼已公開於https://github.com/davidmhart/StyleTransferMasked。

English

Artistic style transfer has long been possible with the advancements of convolution- and transformer-based neural networks. Most algorithms apply the artistic style transfer to the whole image, but individual users may only need to apply a style transfer to a specific region in the image. The standard practice is to simply mask the image after the stylization. This work shows that this approach tends to improperly capture the style features in the region of interest. We propose a partial-convolution-based style transfer network that accurately applies the style features exclusively to the region of interest. Additionally, we present network-internal blending techniques that account for imperfections in the region selection. We show that this visually and quantitatively improves stylization using examples from the SA-1B dataset. Code is publicly available at https://github.com/davidmhart/StyleTransferMasked.