View source for Federated Learning Aggregation Strategies by Weight Exploration

{{StudentProjectTemplate
|Summary=Investigation of aggregation strategies for federated learning
|Keywords=Federated Learning, Aggregation Strategies, Decentralized Learning
|References=Li, Tian, et al. "Federated learning: Challenges, methods, and future directions." IEEE Signal Processing Magazine 37.3 (2020): 50-60.
Kairouz, Peter, et al. "Advances and open problems in federated learning." Foundations and Trends® in Machine Learning 14.1–2 (2021): 1-210.
Rieke, Nicola, et al. "The future of digital health with federated learning." NPJ digital medicine 3.1 (2020): 1-7.
Li, Tian, et al. "Federated optimization in heterogeneous networks." Proceedings of Machine Learning and Systems 2 (2020): 429-450.
|Supervisor=Jens Lundström, Amira Soliman, and Sadi Alawadi
|Examiner=Slawomir Nowaczyk
|Level=Master
|Status=Open
}}
Since the advent of Federated Learning during recent years, industries, institutions and the research community are able to train multiple models on data that stays locally and then aggregate model parameters to form a global model. This set of methods has several advantages including the ability to reduce the burden on a single server instance as well as to preserve privacy which could be valuable when working with highly sensitive data such as Electronic Health Records (EHR). There are several strategies proposed to perform the federated optimization where the most simple is when the server aggregates parameters (e.g. neural network weights) by computing the average of parameters. Past research has shown that despite the effectiveness the simpler federated optimization strategies are sensitive to the computational resource available at each local node and the statistical heterogeneity of the data (since most machine learning methods rely on the data to be identically distributed). Therefore many extensions to the simpler federated optimization have been proposed to measure and communicate the local heterogeneity, which could for some applications breach data privacy. This master thesis is about to study, understand, test and develop innovative and alternative methods based on implicit ways of using the statistical heterogeneity from each local node.