FedRE：面向模型异构联邦学习的表示纠缠框架

FedRE: A Representation Entanglement Framework for Model-Heterogeneous Federated Learning

November 27, 2025

作者: Yuan Yao, Lixu Wang, Jiaqi Wu, Jin Song, Simin Chen, Zehua Wang, Zijian Tian, Wei Chen, Huixia Li, Xiaoxiao Li

cs.AI

摘要

联邦学习（FL）能够在不牺牲隐私的前提下实现跨客户端的协同训练。尽管现有多数联邦学习方法假设采用同构模型架构，但客户端在数据和资源上的异构性使得该假设难以成立，由此催生了模型异构的联邦学习范式。针对该问题，我们提出基于新型客户端知识——纠缠表征（entangled representation）的联邦表征纠缠框架（FedRE）。该框架中，各客户端使用归一化随机权重将本地表征聚合成单一纠缠表征，并应用相同权重将对应独热标签编码整合为纠缠标签编码。这些数据上传至服务器后用于训练全局分类器。训练过程中，每个纠缠表征通过其纠缠标签编码实现跨类别监督，而每轮重新采样的随机权重则引入多样性，有效抑制全局分类器的过置信度并促进更平滑的决策边界。此外，各客户端仅上传单个跨类别纠缠表征及其标签编码，既降低了表征逆推攻击的风险，又减少了通信开销。大量实验表明，FedRE在模型性能、隐私保护和通信开销之间实现了有效平衡。代码已开源：https://github.com/AIResearch-Group/FedRE。

English

Federated learning (FL) enables collaborative training across clients without compromising privacy. While most existing FL methods assume homogeneous model architectures, client heterogeneity in data and resources renders this assumption impractical, motivating model-heterogeneous FL. To address this problem, we propose Federated Representation Entanglement (FedRE), a framework built upon a novel form of client knowledge termed entangled representation. In FedRE, each client aggregates its local representations into a single entangled representation using normalized random weights and applies the same weights to integrate the corresponding one-hot label encodings into the entangled-label encoding. Those are then uploaded to the server to train a global classifier. During training, each entangled representation is supervised across categories via its entangled-label encoding, while random weights are resampled each round to introduce diversity, mitigating the global classifier's overconfidence and promoting smoother decision boundaries. Furthermore, each client uploads a single cross-category entangled representation along with its entangled-label encoding, mitigating the risk of representation inversion attacks and reducing communication overhead. Extensive experiments demonstrate that FedRE achieves an effective trade-off among model performance, privacy protection, and communication overhead. The codes are available at https://github.com/AIResearch-Group/FedRE.