面向视觉-语言数据集蒸馏的排序感知双曲对齐

摘要

视觉语言数据集蒸馏（VLDD）将大规模图像-文本配对数据集压缩为一小组合成配对，从而在严格的数据和计算预算下高效训练对比式视觉语言模型。现有方法大多匹配专家轨迹或跨模态统计量，但仍需在欧几里得嵌入空间中实施全维对齐。由于图像-文本相关性存在秩亏问题，共享语义集中于低维子空间，而剩余变化散布于弱相关的残差子空间中，因此这种全维对齐往往过度严格。LoRS通过低秩分解在相似性层面放宽对齐约束，但未能显式控制表示空间中的主导对齐能力与结构。为此，我们提出一种秩感知双曲对齐方法（RAHA），将层次化几何结构与显式对齐能力控制相结合。RAHA将多模态表示提升至双曲空间，并通过非对称目标优化蒸馏配对：在共享范围内强制实施测地线对齐，同时对残差子空间进行正则化，以保留模态私有多样性并提升迁移鲁棒性。基准实验表明，在固定预算下，RAHA在跨模态检索方面具有竞争力，并改善了迁移指标。

English

Vision-language dataset distillation (VLDD) compresses a large image-text paired dataset into a small set of synthetic pairs that can efficiently train contrastive vision-language models under strict data and compute budgets. Most existing methods match expert trajectories or cross-modal statistics, yet still enforce full-dimensional alignment in a Euclidean embedding space. This is often overly restrictive due to rank-deficient image--text correlation, with shared semantics concentrated in a low-dimensional range and remaining variation spread across a weakly correlated residual subspace. LoRS relaxes alignment at the similarity level by low-rank factorization, but does not explicitly control dominant alignment capacity and structure in the representation space. We thus propose a rank-aware hyperbolic alignment (RAHA) that combines hierarchical geometry with explicit alignment-capacity control. RAHA lifts multimodal representations to hyperbolic space and optimizes distilled pairs with asymmetric objectives that enforce geodesic alignment in the shared range while regularizing the residual subspace to preserve modality-private diversity and improve transfer robustness. Experiments on benchmarks show that RAHA demonstrates competitive cross-modal retrieval and improved transfer indicators under fixed budgets.