基于语义对齐的二维高斯泼溅图像修复方法

摘要

高斯泼溅（Gaussian Splatting, GS）作为一种将离散点转换为连续空间表示的新近技术，在三维场景建模与二维图像超分辨率领域已展现出显著成效。本文中，我们深入探讨了其在图像修复这一要求局部像素合成连贯性与全局语义恢复一致性任务中的未开发潜力。我们首次提出了基于二维高斯泼溅的图像修复框架，该框架将不完整图像编码为二维高斯泼溅系数的连续场，并通过可微分的光栅化过程重建最终图像。高斯泼溅的连续渲染范式本质上促进了修复结果在像素层面的连贯性。为提升效率与可扩展性，我们引入了一种分块光栅化策略，有效降低了内存开销并加速了推理过程。针对全局语义一致性，我们整合了预训练DINO模型的特征。我们发现，DINO的全局特征对小面积缺失区域天然具有鲁棒性，并能有效适应于指导大掩码场景下的语义对齐，确保修复内容与周围场景在上下文上保持一致。在标准基准上的大量实验表明，我们的方法在定量指标与感知质量上均达到了竞争性表现，为高斯泼溅应用于二维图像处理开辟了新的方向。

English

Gaussian Splatting (GS), a recent technique for converting discrete points into continuous spatial representations, has shown promising results in 3D scene modeling and 2D image super-resolution. In this paper, we explore its untapped potential for image inpainting, which demands both locally coherent pixel synthesis and globally consistent semantic restoration. We propose the first image inpainting framework based on 2D Gaussian Splatting, which encodes incomplete images into a continuous field of 2D Gaussian splat coefficients and reconstructs the final image via a differentiable rasterization process. The continuous rendering paradigm of GS inherently promotes pixel-level coherence in the inpainted results. To improve efficiency and scalability, we introduce a patch-wise rasterization strategy that reduces memory overhead and accelerates inference. For global semantic consistency, we incorporate features from a pretrained DINO model. We observe that DINO's global features are naturally robust to small missing regions and can be effectively adapted to guide semantic alignment in large-mask scenarios, ensuring that the inpainted content remains contextually consistent with the surrounding scene. Extensive experiments on standard benchmarks demonstrate that our method achieves competitive performance in both quantitative metrics and perceptual quality, establishing a new direction for applying Gaussian Splatting to 2D image processing.

基于语义对齐的二维高斯泼溅图像修复方法

2D Gaussian Splatting with Semantic Alignment for Image Inpainting

摘要

Support