LoST：面向三维形状的语义层级标记化方法

摘要

分词技术是多模态生成建模中的基础方法，尤其在近期成为3D生成领域重要选择的自回归模型中具有关键作用。然而，如何实现3D形状的最优分词仍是一个悬而未决的问题。现有前沿方法主要依赖最初为渲染和压缩设计的几何细节层次结构，这类空间层次结构往往存在分词效率低下且缺乏自回归建模所需语义一致性的问题。我们提出语义层次分词法（LoST），该方法依据语义显著度对分词进行排序，使得早期前缀可解码为具备主体语义的完整合理形状，而后续分词则用于细化实例特有的几何与语义细节。为训练LoST模型，我们引入了关系性间距对齐（RIDA）——一种新颖的3D语义对齐损失函数，可将3D形状潜空间的关系结构与语义DINO特征空间的关系结构进行对齐。实验表明，LoST在重建任务中达到业界最优水平，在几何与语义重建指标上均大幅超越基于细节层次的3D形状分词方法。此外，LoST仅需先前自回归模型0.1%-10%的分词量即可实现高效、高质量的3D自回归生成，并能支持语义检索等下游任务。

English

Tokenization is a fundamental technique in the generative modeling of various modalities. In particular, it plays a critical role in autoregressive (AR) models, which have recently emerged as a compelling option for 3D generation. However, optimal tokenization of 3D shapes remains an open question. State-of-the-art (SOTA) methods primarily rely on geometric level-of-detail (LoD) hierarchies, originally designed for rendering and compression. These spatial hierarchies are often token-inefficient and lack semantic coherence for AR modeling. We propose Level-of-Semantics Tokenization (LoST), which orders tokens by semantic salience, such that early prefixes decode into complete, plausible shapes that possess principal semantics, while subsequent tokens refine instance-specific geometric and semantic details. To train LoST, we introduce Relational Inter-Distance Alignment (RIDA), a novel 3D semantic alignment loss that aligns the relational structure of the 3D shape latent space with that of the semantic DINO feature space. Experiments show that LoST achieves SOTA reconstruction, surpassing previous LoD-based 3D shape tokenizers by large margins on both geometric and semantic reconstruction metrics. Moreover, LoST achieves efficient, high-quality AR 3D generation and enables downstream tasks like semantic retrieval, while using only 0.1%-10% of the tokens needed by prior AR models.

LoST：面向三维形状的语义层级标记化方法

LoST: Level of Semantics Tokenization for 3D Shapes

摘要

Support