DiffGI：用于高保真薄壳三维生成的可微几何图像

摘要

現有的3D生成模型主要依賴隱式體積表示，這類方法強制模型生成水密拓撲結構，難以處理薄殼與非流形幾何體（如服裝）。基於幾何圖像的方法提供了以表面為核心的替代方案，但現有方法依賴離散的二進制佔用圖，其解析度依賴的邊界編碼在下採樣時會造成階梯偽影與資訊損失，且表面重建作為不可微分的後處理步驟，與學習流程脫節。為解決此問題，我們提出可微分幾何圖像（Differentiable Geometry Image, DiffGI），這是一個端到端的3D轉2D映射框架，可無縫整合表面表示與幾何優化。DiffGI以連續的2D截斷符號距離函數（TSDF）取代二進制圖，在固定網格解析度下以子像素精度編碼邊界位置，即使經過劇烈下採樣也能消除依賴解析度的階梯偽影。在此連續場的基礎上，我們引入基於解析線性插值的可微分移動平方演算法，使得來自3D表面損失的梯度能夠反向傳播至2D潛在空間。利用此可微分管線，我們訓練了結合幾何感知法向渲染損失的DiffGI-VAE，將複雜的3D表面壓縮至超緊湊的32×32潛在空間，並在此空間上基於流匹配目標實例化了一個基於Transformer的潛在擴散模型，用於條件式3D生成。在服裝與物體資料集上的廣泛實驗證明，相較於先前的幾何圖像與體積基方法，我們的方法在重建保真度與邊界精確度上表現更優，同時所需的計算資源顯著更少。

English

Existing 3D generative models predominantly rely on implicit volumetric representations, which enforce watertight topology and struggle to represent thin-shell and non-manifold geometries such as garments. Geometry image-based approaches offer a surface-centric alternative, but existing methods rely on discrete binary occupancy maps whose resolution-dependent boundary encoding causes staircase artifacts and information loss upon downsampling, while surface reconstruction remains a non-differentiable post-processing step disconnected from the learning pipeline. To address this, we propose Differentiable Geometry Image (DiffGI), an end-to-end 3D-to-2D mapping framework that seamlessly integrates surface representation and geometric optimization. DiffGI replaces binary maps with a continuous 2D Truncated Signed Distance Function (TSDF), which encodes boundary position at subpixel precision within a fixed grid resolution, eliminating resolution-dependent staircase artifacts even under aggressive downsampling. Building on this continuous field, we introduce a differentiable Marching Squares algorithm based on analytical linear interpolation, allowing gradients from 3D surface losses to propagate back to the 2D latent space. Leveraging this differentiable pipeline, we train a DiffGI-VAE augmented with a geometry-aware normal rendering loss to compress complex 3D surfaces into an ultra-compact 32X32 latent space, and instantiate a transformer-based latent diffusion model with a flow-matching objective on top of this space for conditional 3D generation. Extensive experiments on garment and object datasets demonstrate that our method achieves superior reconstruction fidelity and boundary precision compared to prior geometry-image and voxel-based approaches, while requiring significantly fewer computational resources.