Citation: | Alessandro Giuseppi, Sabato Manfredi, Antonio Pietrabissa. A Weighted Average Consensus Approach for Decentralized Federated Learning. Machine Intelligence Research, vol. 19, no. 4, pp.319-330, 2022. https://doi.org/10.1007/s11633-022-1338-z |
[1] |
H. B. McMahan, E. Moore, D. Ramage, S. Hampson. Communication-efficient learning of deep networks from decentralized data. In Proceedings of the 20th International Conference on Artificial Intelligence and Statistics, Fort Lauderdale, USA, pp. 1273–1282, 2016.
|
[2] |
T. Li, A. K. Sahu, M. Zaheer, M. Sanjabi, A. Talwalkar, V. Smith. Federated optimization in heterogeneous networks. In Proceedings of the 3rd Conference on Machine learning and System, Austin, USA, pp. 429–450, 2018.
|
[3] |
H. B. McMahan, E. Moore, D. Ramage, B. A. Y. Arcas. Federated learning of deep networks using model averaging. [Online], Available: https://arxiv.org/abs/1602.05629, 2016.
|
[4] |
Y. F. Ye, S. Li, F. Liu, Y. H. Tang, W. T. Hu. EdgeFed: Optimized federated learning based on edge computing. IEEE Access, vol. 8, pp. 209191–209198, 2020. DOI: 10.1109/access.2020.3038287.
|
[5] |
L. U. Khan, M. Alsenwi, I. Yaqoob, M. Imran, Z. Han, C. S. Hong. Resource optimized federated learning-enabled cognitive internet of things for smart industries. IEEE Access, vol. 8, pp. 168854–168864, 2020. DOI: 10.1109/access.2020.3023940.
|
[6] |
A. Hard, K. Rao, R. Mathews, S. Ramaswamy, F. Beaufays, S. Augenstein, H. Eichner, C. Kiddon, D. Ramage. Federated learning for mobile keyboard prediction. [Online], Available: https://arxiv.org/abs/1811.03604, 2018.
|
[7] |
T. Yang, G. Andrew, H. Eichner, H. C. Sun, W. Li, N. Kong, D. Ramage, F. Beaufays. Applied federated learning: Improving Google keyboard query suggestions. [Online], Available: https://arxiv.org/abs/1812.02903, 2018.
|
[8] |
S. Ramaswamy, R. Mathews, K. Rao, F. Beaufays. Federated learning for Emoji prediction in a mobile keyboard. [Online], Available: https://arxiv.org/abs/1906.04329, 2019.
|
[9] |
J. H. Luo, X. Y. Wu, Y. Luo, A. B. Huang, Y. F. Huang, Y. Liu, Q. Yang. Real-world image datasets for federated learning. [Online], Available: https:/arxiv.org/abs/1910.11089, 2019.
|
[10] |
L. Ahmed, K. Ahmad, N. Said, B. Qolomany, J. Qadir, A. Al-Fuqaha. Active learning based federated learning for waste and natural disaster image classification. IEEE Access, vol. 8, pp. 208518–208531, 2020. DOI: 10.1109/access.2020.3038676.
|
[11] |
T. S. Brisimi, R. D. Chen, T. Mela, A. Olshevsky, I. C. Paschalidis, W. Shi. Federated learning of predictive models from federated electronic health records. International Journal of Medical Informatics, vol. 112, pp. 59–67, 2018. DOI: 10.1016/j.ijmedinf.2018.01.007.
|
[12] |
M. J. Sheller, G. A. Reina, B. Edwards, J. Martin, S. Bakas. Multi-institutional deep learning modeling without sharing patient data: A feasibility study on brain tumor segmentation. In Proceedings of the 4th International Workshop on Brainlesion: Glioma, Multiple Sclerosis, Stroke and Traumatic Brain Injuries, Springer, Granada, Spain, pp. 92–104, 2019. DOI: 10.1007/978-3-030-11723-8_9.
|
[13] |
M. Aledhari, R. Razzak, R. M. Parizi, F. Saeed. Federated learning: A survey on enabling technologies, protocols, and applications. IEEE Access, vol. 8, pp. 140699–140725, 2020. DOI: 10.1109/access.2020.3013541.
|
[14] |
H. L. Yang, J. Zhao, Z. H. Xiong, K. Y. Lam, S. M. Sun, L. Xiao. Privacy-preserving federated learning for UAV-enabled networks: Learning-based joint scheduling and resource management. IEEE Journal on Selected Areas in Communications, vol. 39, no. 10, pp. 3144–3159, 2021. DOI: 10.1109/JSAC.2021.3088655.
|
[15] |
M. Hao, H. W. Li, X. Z. Luo, G. W. Xu, H. M. Yang, S. Liu. Efficient and privacy-enhanced federated learning for industrial artificial intelligence. IEEE Transactions on Industrial Informatics, vol. 16, no. 10, pp. 6532–6542, 2020. DOI: 10.1109/TII.2019.2945367.
|
[16] |
K. Wei, J. Li, C. Ma, M. Ding, S. Wei, F. Wu, G. H. Chen, T. Ranbaduge. Vertical federated learning: Challenges, methodologies and experiments. [Online], Available: https://arxiv.org/abs/2202.04309, 2022.
|
[17] |
Q. B. Li, Z. Y. Wen, Z. M. Wu, S. X. Hu, N. B. Wang, Y. Li, X. Liu, B. S. He. A survey on federated learning systems: Vision, hype and reality for data privacy and protection. IEEE Transactions on Knowledge and Data Engineering, to be published. DOI: 10.1109/TKDE.2021.3124599.
|
[18] |
W. Y. B. Lim, N. C. Luong, D. T. Hoang, Y. T. Jiao, Y. C. Liang, Q. Yang, D. Niyato, C. Y. Miao. Federated learning in mobile edge networks: A comprehensive survey. IEEE Communications Surveys &Tutorials, vol. 22, no. 3, pp. 2031–2063, 2020. DOI: 10.1109/COMST.2020.2986024.
|
[19] |
K. Bonawitz, V. Ivanov, B. Kreuter, A. Marcedone, H. B. McMahan, S. Patel, D. Ramage, A. Segal, K. Seth. Practical secure aggregation for privacy-preserving machine learning. In Proceedings of ACM SIGSAC Conference on Computer and Communications Security, ACM, Dallas, USA, pp. 1175–1191, 2017. DOI: 10.1145/3133956.3133982.
|
[20] |
R. C. Geyer, T. Klein, M. Nabi. Differentially private federated learning: A client level perspective. [Online], Available: https://arxiv.org/abs/1712.07557, 2017.
|
[21] |
S. Truex, N. Baracaldo, A. Anwar, T. Steinke, H. Ludwig, R. Zhang, Y. Zhou. A hybrid approach to privacy-preserving federated learning. In Proceedings of the 12th ACM Workshop on Artificial Intelligence and Security, ACM, London, USA, pp. 1–11, 2019. DOI: 10.1145/3338501.3357370.
|
[22] |
Q. S. Zhang, B. Gu, C. Deng, H. Huang. Secure bilevel asynchronous vertical federated learning with backward updating. In Proceedings of AAAI Conference on Artificial Intelligence, vol. 35, no. 12, pp. 10896–10904, 2021.
|
[23] |
Z. B. Wang, M. K. Song, Z. F. Zhang, Y. Song, Q. Wang, H. R. Qi. Beyond inferring class representatives: User-level privacy leakage from federated learning. In Proceedings of IEEE Conference on Computer Communications, IEEE, Paris, France, pp. 2512–2520, 2019. DOI: 10.1109/infocom.2019.8737416.
|
[24] |
S. Kim. Incentive design and differential privacy based federated learning: A mechanism design perspective. IEEE Access, vol. 8, pp. 187317–187325, 2020. DOI: 10.1109/access.2020.3030888.
|
[25] |
B. Gu, A. Xu, Z. Y. Huo, C. Deng, H. Huang. Privacy-preserving asynchronous federated learning algorithms for multi-party vertically collaborative learning. [Online], Available: https://arxiv.org/abs/2008.06233, 2020.
|
[26] |
J. Konečný, H. B. McMahan, F. X. Yu, P. Richtárik, A. T. Suresh, D. Bacon. Federated learning: Strategies for improving communication efficiency. [Online], Available: https:/arxiv.org/abs/1610.05492, 2016.
|
[27] |
F. Sattler, S. Wiedemann, K. R. Müller, W. Samek. Robust and communication-efficient federated learning from Non-i.i.d. data. IEEE Transactions on Neural Networks and Learning Systems, vol. 31, no. 9, pp. 3400–3413, 2020. DOI: 10.1109/tnnls.2019.2944481.
|
[28] |
Q. S. Zhang, B. Gu, C. Deng, S. X. Gu, L. F. Bo, J. Pei, H. Huang. AsySQN: Faster vertical federated learning algorithms with better computation resource utilization. In Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery & Data Mining, ACM, Washington DC, USA, pp. 3917–3927, 2021. DOI: 10.1145/3447548.3467169.
|
[29] |
Q. S. Zhang, B. Gu, Z. Y. Dang, C. Deng, H. Huang. Desirable companion for vertical federated learning: New Zeroth-order gradient based algorithm. In Proceedings of the 30th ACM International Conference on Information & Knowledge Management, ACM, Atlanta, USA, pp. 2598–2607, 2021. DOI: 10.1145/3459637.3482249.
|
[30] |
C. H. Hu, J. Y. Jiang, Z. Wang. Decentralized federated learning: A segmented gossip approach. [Online], Available: https://arxiv.org/abs/1908.07782, 2019.
|
[31] |
J. Y. Jiang, L. Hu. Decentralised federated learning with adaptive partial gradient aggregation. CAAI Transactions on Intelligence Technology, vol. 5, no. 3, pp. 230–236, 2020. DOI: 10.1049/trit.2020.0082.
|
[32] |
X. H. Chen, J. L. Ji, C. Q. Luo, W. X. Liao, P. Li. When machine learning meets blockchain: A decentralized, privacy-preserving and secure design. In Proceedings of IEEE International Conference on Big Data, IEEE, Seattle, USA, pp. 1178–1187, 2018. DOI: 10.1109/BigData.2018.8622598.
|
[33] |
Y. Z. Li, C. Chen, N. Liu, H. W. Huang, Z. B. Zheng, Q. Yan. A blockchain-based decentralized federated learning framework with committee consensus. IEEE Network, vol. 35, no. 1, pp. 234–241, 2021. DOI: 10.1109/MNET.011.2000263.
|
[34] |
A. Nedic. Distributed gradient methods for convex machine learning problems in networks: Distributed optimization. IEEE Signal Processing Magazine, vol. 37, no. 3, pp. 92–101, 2020. DOI: 10.1109/MSP.2020.2975210.
|
[35] |
S. Manfredi, D. Angeli. Robust distributed estimation of the maximum of a field. IEEE Transactions on Control of Network Systems, vol. 7, no. 1, pp. 372–383, 2020. DOI: 10.1109/TCNS.2019.2906865.
|
[36] |
B. D. O. Anderson, M. B. Ye. Recent advances in the modelling and analysis of opinion dynamics on influence networks. International Journal of Automation and Computing, vol. 16, no. 2, pp. 129–149, 2019. DOI: 10.1007/s11633-019-1169-8.
|
[37] |
W. Ren. Consensus based formation control strategies for multi-vehicle systems. In Proceedings of American Control Conference, IEEE, Minneapolis, USA, pp. 4237–4242, 2006. DOI: 10.1109/ACC.2006.1657384.
|
[38] |
Z. A. Zhang, M. Y. Chow. Convergence analysis of the incremental cost consensus algorithm under different communication network topologies in a smart grid. IEEE Transactions on Power Systems, vol. 27, no. 4, pp. 1761–1768, 2012. DOI: 10.1109/TPWRS.2012.2188912.
|
[39] |
C. T. Dinh, N. H. Tran, M. N. H. Nguyen, C. S. Hong, W. Bao, A. Y. Zomaya, V. Gramoli. Federated learning over wireless networks: Convergence analysis and resource allocation. IEEE/ACM Transactions on Networking, vol. 29, no. 1, pp. 398–409, 2021. DOI: 10.1109/tnet.2020.3035770.
|
[40] |
R. Olfati-Saber, R. M. Murray. Consensus problems in networks of agents with switching topology and time-delays. IEEE Transactions on Automatic Control, vol. 49, no. 9, pp. 1520–1533, 2004. DOI: 10.1109/TAC.2004.834113.
|
[41] |
F. Pedroche, M. Rebollo, C. Carrascosa, A. Palomares. Convergence of weighted-average consensus for undirected graphs. International Journal of Complex Systems in Science, vol. 4, no. 1, pp. 13–16, 2014.
|
[42] |
K. Ogata. Discrete-time Control Systems, 2nd ed., New York, USA: Prentice-Hall, Inc., 1995.
|
[43] |
F. Haddadpour, M. M. Kamani, A. Mokhtari, M. Mahdavi. Federated learning with compression: Unified analysis and sharp guarantees. In Proceedings of the 24th International Conference on Artificial Intelligence and Statistics, Beijing, China, pp. 2350–2358, 2021.
|
[44] |
A. Albasyoni, M. Safaryan, L. Condat, P. Richtárik. Optimal gradient compression for distributed and federated learning. [Online], Available: https://arxiv.org/abs/2010.03246, 2020.
|
[45] |
L. P. Wang, W. Wang, B. Li. CMFL: Mitigating communication overhead for federated learning. In Proceedings of the 39th IEEE International Conference on Distributed Computing Systems, IEEE, Dallas, USA, pp. 954–964, 2019. DOI: 10.1109/icdcs.2019.00099.
|
[46] |
Y. LeCun, C. Cortes, C. J. C. Burges. MNIST handwritten digit database. ATT Labs. [Online], Available: http://yann.lecun.com/exdb/mnist, 2010.
|
[47] |
A. Krizhevsky. Learning Multiple Layers of Features from Tiny Images, Technical Report TR-2009, University of Toronto, Toronto, Canada, 2009.
|
[48] |
M. Shaha, M. Pawar. Transfer learning for image classification. In Proceedings of the 2nd International Conference on Electronics, Communication and Aerospace Technology, IEEE, Coimbatore, India, pp. 656–660, 2018. DOI: 10.1109/ICECA.2018.8474802.
|
[49] |
M. Huh, P. Agrawal, A. A. Efros. What makes ImageNet good for transfer learning? [Online], Available: https://arxiv.org/abs/1608.08614, 2016.
|