FLUX3D：基於擴散對齊稀疏表示的高保真3D高斯生成

摘要

稀疏體素表示已成為圖像到3D高斯潑濺（3DGS）生成的可擴展基礎，然而現有方法由於兩個結構性瓶頸，難以保留輸入圖像的高頻視覺細節。首先，這些方法採用針對語義抽象優化的判別式2D特徵來構建稀疏體素潛變量，壓抑了重建線索並引發表徵瓶頸。其次，在生成階段，標準擴散變壓器缺乏有效機制將密集2D圖像標記與稀疏3D體素潛變量對齊，導致跨模態對應瓶頸。為解決這些問題，我們提出FLUX3D——一個可擴展的圖像到3DGS框架，在生成過程中同時提升表徵學習與跨模態對齊。我們首先重新審視基於稀疏體素的3D表徵學習中的2D特徵選擇，提出擴散對齊結構化潛變量（DA-SLAT）並將其與僅解碼器架構結合，以提升3DGS重建保真度。我們還設計了一個稀疏結構感知擴散框架，整合稀疏結構多模態擴散變壓器（SMDiT）與模態感知旋轉位置嵌入（MARoPE），實現幾何無關的2D-3D對齊。廣泛的基準實驗表明，FLUX3D在外觀保真度上實現顯著改進，並在生成高品質3DGS資產方面全面優於所有最先進（SOTA）方法。

English

Sparse voxel representation has emerged as a scalable foundation for image-to-3D Gaussian Splatting (3DGS) generation, yet current methods struggle to preserve high-frequency visual details of input images due to two structural bottlenecks. First, they adopt discriminative 2D features optimized for semantic abstraction to construct sparse voxel latents, which suppress reconstructive cues and induce a representation bottleneck. Second, in the generation stage, standard diffusion transformers lack effective mechanisms to align dense 2D image tokens with sparse 3D voxel latents, resulting in a cross-modal correspondence bottleneck. To address these issues, we propose FLUX3D, a scalable image-to-3DGS framework that boosts both representation learning and cross-modal alignment during generation. We first revisit 2D feature selection for sparse-voxel-based 3D representation learning, propose Diffusion-Aligned Structured Latents (DA-SLAT) and couple it with a decoder-only architecture to improve 3DGS reconstruction fidelity. We also design a sparse-structure-aware diffusion framework, which integrates the Sparse-structure Multimodal Diffusion Transformer (SMDiT) and Modal-Aware Rotary Positional Embedding (MARoPE) to achieve geometry-agnostic 2D-3D alignment. Extensive benchmark experiments demonstrate that FLUX3D yields substantial improvements in appearance fidelity and significantly outperforms all state-of-the-art (SOTA) methods in generating high-quality 3DGS assets.