DiffGI：面向高保真薄壳3D生成的可微分几何图像

摘要

现有的3D生成模型主要依赖隐式体积表示，此类方法强制要求水密拓扑结构，难以表达薄壳或非流形几何（如服装）。基于几何图像的方法提供了一种以表面为中心的替代方案，但现有方法依赖离散的二值占据图，其分辨率依赖的边界编码在下采样时会导致阶梯伪影和信息损失，同时表面重建仍是一个与学习流程脱节的不可微后处理步骤。为解决这些问题，我们提出可微几何图像（DiffGI），这是一个无缝集成表面表示与几何优化的端到端3D到2D映射框架。DiffGI用连续的二维截断符号距离函数（TSDF）替代二值图，该函数在固定网格分辨率下以亚像素精度编码边界位置，即使进行大幅度下采样也能消除与分辨率相关的阶梯伪影。基于这一连续场，我们引入基于解析线性插值的可微行进方块算法，使3D表面损失的梯度能够反向传播到2D潜空间。借助这一可微流水线，我们训练了DiffGI-VAE模型，并辅以几何感知的法线渲染损失，将复杂3D表面压缩至超紧凑的32×32潜空间；在此潜空间上，我们通过流匹配目标实例化了一个基于Transformer的潜扩散模型，用于条件3D生成。在服装和物体数据集上的大量实验表明，与先前的几何图像和体素方法相比，我们的方法在重建保真度和边界精度上表现更优，同时显著降低了计算资源需求。

English

Existing 3D generative models predominantly rely on implicit volumetric representations, which enforce watertight topology and struggle to represent thin-shell and non-manifold geometries such as garments. Geometry image-based approaches offer a surface-centric alternative, but existing methods rely on discrete binary occupancy maps whose resolution-dependent boundary encoding causes staircase artifacts and information loss upon downsampling, while surface reconstruction remains a non-differentiable post-processing step disconnected from the learning pipeline. To address this, we propose Differentiable Geometry Image (DiffGI), an end-to-end 3D-to-2D mapping framework that seamlessly integrates surface representation and geometric optimization. DiffGI replaces binary maps with a continuous 2D Truncated Signed Distance Function (TSDF), which encodes boundary position at subpixel precision within a fixed grid resolution, eliminating resolution-dependent staircase artifacts even under aggressive downsampling. Building on this continuous field, we introduce a differentiable Marching Squares algorithm based on analytical linear interpolation, allowing gradients from 3D surface losses to propagate back to the 2D latent space. Leveraging this differentiable pipeline, we train a DiffGI-VAE augmented with a geometry-aware normal rendering loss to compress complex 3D surfaces into an ultra-compact 32X32 latent space, and instantiate a transformer-based latent diffusion model with a flow-matching objective on top of this space for conditional 3D generation. Extensive experiments on garment and object datasets demonstrate that our method achieves superior reconstruction fidelity and boundary precision compared to prior geometry-image and voxel-based approaches, while requiring significantly fewer computational resources.