通過幾何約束改善不平衡回歸中的表示學習

Improve Representation for Imbalanced Regression through Geometric Constraints

March 2, 2025

作者: Zijian Dong, Yilei Wu, Chongyao Chen, Yingtian Zou, Yichi Zhang, Juan Helen Zhou

cs.AI

摘要

在表徵學習中，均勻性指的是潛在空間（即單位超球面）內特徵的均勻分佈。先前的研究表明，提升均勻性有助於學習那些代表性不足的類別。然而，大多數先前的工作主要集中在分類問題上；對於不平衡迴歸的表徵空間仍未被探索。基於分類的方法不適用於迴歸任務，因為它們將特徵聚類成不同的組別，而沒有考慮到迴歸所必需的連續性和有序性。從幾何角度出發，我們獨特地專注於通過兩個關鍵損失來確保不平衡迴歸在潛在空間中的均勻性：包絡損失和同質性損失。包絡損失促使誘導的軌跡均勻地佔據超球面的表面，而同質性損失則確保平滑性，使表徵在一致的間隔下均勻分佈。我們的方法通過一個代理驅動的表徵學習（SRL）框架，將這些幾何原理整合到數據表徵中。在真實世界的迴歸和運算元學習任務中的實驗，突顯了均勻性在不平衡迴歸中的重要性，並驗證了我們基於幾何的損失函數的有效性。

English

In representation learning, uniformity refers to the uniform feature distribution in the latent space (i.e., unit hypersphere). Previous work has shown that improving uniformity contributes to the learning of under-represented classes. However, most of the previous work focused on classification; the representation space of imbalanced regression remains unexplored. Classification-based methods are not suitable for regression tasks because they cluster features into distinct groups without considering the continuous and ordered nature essential for regression. In a geometric aspect, we uniquely focus on ensuring uniformity in the latent space for imbalanced regression through two key losses: enveloping and homogeneity. The enveloping loss encourages the induced trace to uniformly occupy the surface of a hypersphere, while the homogeneity loss ensures smoothness, with representations evenly spaced at consistent intervals. Our method integrates these geometric principles into the data representations via a Surrogate-driven Representation Learning (SRL) framework. Experiments with real-world regression and operator learning tasks highlight the importance of uniformity in imbalanced regression and validate the efficacy of our geometry-based loss functions.