用於視覺-語言數據集蒸餾的排序感知雙曲對齊

摘要

視覺語言資料集蒸餾（VLDD）將大型圖像與文字配對的資料集壓縮成少量合成配對，使對比式視覺語言模型能在嚴格的資料與計算預算下高效訓練。現有方法大多匹配專家軌跡或跨模態統計，但仍強制在歐幾里得嵌入空間中進行全維度對齊。由於圖像與文字之間存在秩虧損相關性，其共享語義集中於低維度範圍，而剩餘變異則分散於弱相關的殘差子空間，因此這種對齊方式往往過於嚴格。LoRS透過低秩分解在相似度層級放寬對齊，但未明確控制表徵空間中的主導對齊能力與結構。為此，我們提出秩感知雙曲對齊（RAHA），結合層級幾何結構與明確的對齊能力控制。RAHA將多模態表徵提升至雙曲空間，並以不對稱目標優化蒸餾配對，在共享範圍內強制測地線對齊，同時正則化殘差子空間以保留模態私有多樣性並提升遷移穩健性。基準實驗顯示，RAHA在固定預算下展現具競爭力的跨模態檢索與改善的遷移指標。

English

Vision-language dataset distillation (VLDD) compresses a large image-text paired dataset into a small set of synthetic pairs that can efficiently train contrastive vision-language models under strict data and compute budgets. Most existing methods match expert trajectories or cross-modal statistics, yet still enforce full-dimensional alignment in a Euclidean embedding space. This is often overly restrictive due to rank-deficient image--text correlation, with shared semantics concentrated in a low-dimensional range and remaining variation spread across a weakly correlated residual subspace. LoRS relaxes alignment at the similarity level by low-rank factorization, but does not explicitly control dominant alignment capacity and structure in the representation space. We thus propose a rank-aware hyperbolic alignment (RAHA) that combines hierarchical geometry with explicit alignment-capacity control. RAHA lifts multimodal representations to hyperbolic space and optimizes distilled pairs with asymmetric objectives that enforce geodesic alignment in the shared range while regularizing the residual subspace to preserve modality-private diversity and improve transfer robustness. Experiments on benchmarks show that RAHA demonstrates competitive cross-modal retrieval and improved transfer indicators under fixed budgets.