Federated Learning (FL), which serves as a distributed learning model focused on privacy protection, has received widespread attention due to its ability to collaboratively train models without exchanging clients’ raw data. However, since each client independently collects its local dataset, a single client may contain data from multiple domains, which leads to the existence of an agnostic distribution bias both intra-client and inter-client biases, and it is still a challenge to implement cross-domain generalization of the model in such an FL setting. In this paper, we propose a new stable federated adversarial domain generalization (Stable-FedADG) algorithm, which considers both intra-client and inter-client multi-domain distributions and through intra-client stable feature learning and inter-client domain adversarial alignment to learn domain-invariant features in federated scenarios. Experiments verify that the Stable-FedADG algorithm enhances the overall model performance across various federated environments.
References
[1]
Xu, H., Seng, K.P., Ang, L.M. and Smith, J. (2024) Decentralized and Distributed Learning for AIoT: A Comprehensive Review, Emerging Challenges, and Opportunities. IEEEAccess, 12, 101016-101052. https://doi.org/10.1109/access.2024.3422211
[2]
Li, Y., Wang, X., Zeng, R., Yang, M., Li, K., Huang, M., etal. (2023) VARF: An Incentive Mechanism of Cross-Silo Federated Learning in Mec. IEEEInternetofThingsJournal, 10, 15115-15132. https://doi.org/10.1109/jiot.2023.3264611
[3]
Liu, B., Lv, N., Guo, Y. and Li, Y. (2024) Recent Advances on Federated Learning: A Systematic Survey. Neurocomputing, 597, Article 128019. https://doi.org/10.1016/j.neucom.2024.128019
[4]
McMahan, B., Moore, E., Ramage, D., Hampson, S. and y Arcas, B.A. (2017) Communication-Efficient Learning of Deep Networks from Decentralized Data. Proceedings of the 20th International Conference on Artificial Intelligence and Statistics (AISTATS), Fort Lauderdale, 20-22 April 2017, 1273-1282. https://api.semanticscholar.org/CorpusID:14955348
[5]
Ye, M., Fang, X., Du, B., Yuen, P.C. and Tao, D. (2023) Heterogeneous Federated Learning: State-of-the-Art and Research Challenges. ACMComputingSurveys, 56, 1-44. https://doi.org/10.1145/3625558
[6]
Lu, Z., Pan, H., Dai, Y., Si, X. and Zhang, Y. (2024) Federated Learning with Non-IID Data: A Survey. IEEEInternetofThingsJournal, 11, 19188-19209. https://doi.org/10.1109/jiot.2024.3376548
[7]
Li, T., Sahu, A.K., Zaheer, M., Sanjabi, M., Talwalkar, A. and Smith, V. (2020) Federated Optimization in Heterogeneous Networks. Proceedings of the 3rd MLSys Conference, Austin, 2-4 March 2020, 429-450. https://proceedings.mlsys.org/paper_files/paper/2020/file/1f5fe83998a09396ebe6477d9475ba0c-Paper.pdf
[8]
Karimireddy, S.P., Kale, S., Mohri, M., Reddi, S., Stich, S. and Suresh, A.T. (2020) Scaffold: Stochastic Controlled Averaging for Federated Learning. Proceedingsofthe 37thInternationalConferenceonMachineLearning, Virtual Event, 13-18 July 2020, 5132-5143. http://proceedings.mlr.press/v119/karimireddy20a.html
[9]
Tan, A.Z., Yu, H., Cui, L. and Yang, Q. (2023) Towards Personalized Federated Learning. IEEETransactionsonNeuralNetworksandLearningSystems, 34, 9587-9603. https://doi.org/10.1109/tnnls.2022.3160699
[10]
Lin, S., Yang, G. and Zhang, J. (2020) A Collaborative Learning Framework via Federated Meta-Learning. 2020 IEEE 40th International Conference on Distributed ComputingSystems (ICDCS), Singapore, 29 November-1 December 2020, 289-299. https://doi.org/10.1109/icdcs47774.2020.00032
[11]
Li, Y., Wang, X., Zeng, R., Donta, P.K., Murturi, I., Huang, M. and Dustdar, S. (2024) Federated Domain Generalization: A Survey. arXiv: 2306.01334. https://arxiv.org/abs/2306.01334
[12]
Li, X., Gu, Y., Dvornek, N., Staib, L.H., Ventola, P. and Duncan, J.S. (2020) Multi-Site fMRI Analysis Using Privacy-Preserving Federated Learning and Domain Adaptation: ABIDE Results. MedicalImageAnalysis, 65, Article 101765. https://doi.org/10.1016/j.media.2020.101765
[13]
Peng, X., Huang, Z., Zhu, Y. and Saenko, K. (2020) Federated Adversarial Domain Adaptation. https://openreview.net/forum?id=HJezF3VYPB
[14]
Wu, G. and Gong, S. (2021) Collaborative Optimization and Aggregation for Decentralized Domain Generalization and Adaptation. 2021 IEEE/CVFInternationalConferenceonComputerVision (ICCV), Montreal, 10-17 October 2021, 6464-6473. https://doi.org/10.1109/iccv48922.2021.00642
[15]
Liu, Q., Chen, C., Qin, J., Dou, Q. and Heng, P. (2021) FedDG: Federated Domain Generalization on Medical Image Segmentation via Episodic Learning in Continuous Frequency Space. 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Nashville, 20-25 June 2021, 1013-1023. https://doi.org/10.1109/cvpr46437.2021.00107
[16]
Zhang, L., Lei, X., Shi, Y., Huang, H. and Chen, C. (2023) Federated Learning for IoT Devices with Domain Generalization. IEEE Internet of Things Journal, 10, 9622-9633. https://doi.org/10.1109/jiot.2023.3234977
[17]
Shen, Z., Cui, P., Kuang, K., Li, B. and Chen, P. (2018) Causally Regularized Learning with Agnostic Data Selection Bias. Proceedingsofthe 26thACMinternationalconferenceonMultimedia, Seoul, 22-26 October 2018, 411-419. https://doi.org/10.1145/3240508.3240577
[18]
Kuang, K., Xiong, R., Cui, P., Athey, S. and Li, B. (2020) Stable Prediction with Model Misspecification and Agnostic Distribution Shift. ProceedingsoftheAAAIConferenceonArtificialIntelligence, 34, 4485-4492. https://doi.org/10.1609/aaai.v34i04.5876
[19]
Cui, P. and Athey, S. (2022) Stable Learning Establishes Some Common Ground between Causal Inference and Machine Learning. NatureMachineIntelligence, 4, 110-115. https://doi.org/10.1038/s42256-022-00445-z
[20]
Xu, R., Cui, P., Shen, Z., Zhang, X. and Zhang, T. (2021) Why Stable Learning Works? A Theory of Covariate Shift Generalization. arXiv: 2111.02355. https://arxiv.org/abs/2111.02355v1
[21]
Zhang, X., Cui, P., Xu, R., Zhou, L., He, Y. and Shen, Z. (2021) Deep Stable Learning for Out-of-Distribution Generalization. 2021 IEEE/CVFConferenceonComputerVisionandPatternRecognition (CVPR), Nashville, 20-25 June 2021, 5368-5378. https://doi.org/10.1109/cvpr46437.2021.00533
[22]
Guo, Y., Guo, K., Cao, X., Wu, T. and Chang, Y. (2023) Out-of-Distribution Generalization of Federated Learning via Implicit Invariant Relationships. Proceedings of the 40th International Conference on Machine Learning, Honolulu, 23-29 July 2023, 11905-11933. https://dl.acm.org/doi/10.5555/3618408.3618886
[23]
Nguyen, A.T., Torr, P. and Lim, S.N. (2022) FedSR: A Simple and Effective Domain Generalization Method for Federated Learning. Proceedings of the 36th International Conference on Neural Information Processing System, New Orleans, 28 November-9 December 2022, 38831-38843. https://dl.acm.org/doi/10.5555/3600270.3603084
