Staleness-Controlled Asynchronous Federated Learning: Accuracy and Efficiency Tradeoff

doi:10.1109/TMC.2024.3416216

	Staleness-Controlled Asynchronous Federated Learning: Accuracy and Efficiency Tradeoff
	Sun, Sheng 1; Zhang, Zengqi 2; Pan, Quyang 3,4; Liu, Min 3; Wang, Yuwei 1; He, Tianliu 3,4; Chen, Yali 1; Wu, Zhiyuan 3,4
	2024-12-01
发表期刊	IEEE TRANSACTIONS ON MOBILE COMPUTING
ISSN	1536-1233
卷号	23 期号:12 页码:12621-12634
摘要	Federated Learning (FL) is an emerging distributed learning paradigm with the privacy-preserving advantage of collaboratively training a shared model across multiple participants. Considering the prevailing device heterogeneity circumstance in practice, asynchronous interaction is introduced into FL to break the straggler barrier of synchronization, at the cost of significant accuracy degradation derived from model staleness. Although quite a few works attempt to partially mitigate the detrimental impact after occurring staleness issue, they neglect to control the overall staleness degree of clients-side local models from the whole training perspective, resulting in highly-stale models for aggregation and slow convergence speed. To this end, we propose a Staleness-Controlled Asynchronous Federated Learning (SC-AFL) method, which enables to restrict staleness degree of local models within a certain bound via dynamically tuning the aggregated strategy of each round, aiming to strike a good balance between accuracy guarantee and convergence acceleration. Specifically, we leverage the Lyapunov optimization framework to decouple the troublesome round-coupling problem into the single-round sequential solving problem, and further develop a deterministic algorithm that selects the aggregated number of clients to minimize training time under the constraint of maintaining staleness queue stability. Besides, we derive the theoretical convergence analysis of SC-AFL and also present the upper bound of the performance gap with the optimum. Extensive experiments on three datasets demonstrate the superiority of SC-AFL in terms of time-to-accuracy speedup on both IID and Non-IID data distributions, achieving a good balance between model accuracy and convergence efficiency in AFL system.
关键词	Asynchronous learning convergence efficiency federated learning lyapunov optimization model accuracy Asynchronous learning convergence efficiency federated learning lyapunov optimization model accuracy
DOI	10.1109/TMC.2024.3416216
收录类别	SCI
语种	英语
资助项目	National Key Research and Development Program of China[2021YFB2900102] ; National Natural Science Foundation of China[62072436] ; National Natural Science Foundation of China[62202449]
WOS研究方向	Computer Science ; Telecommunications
WOS类目	Computer Science, Information Systems ; Telecommunications
WOS记录号	WOS:001359244600140
出版者	IEEE COMPUTER SOC
引用统计
文献类型	期刊论文
条目标识符	http://119.78.100.204/handle/2XEOYT63/41107
专题	中国科学院计算技术研究所期刊论文_英文
通讯作者	Liu, Min
作者单位	1.Chinese Acad Sci, Inst Comp Technol, Beijing 100190, Peoples R China 2.Huawei Technol Co Ltd, Beijing 100083, Peoples R China 3.Chinese Acad Sci, Inst Comp Technol, Beijing 100190, Peoples R China 4.Univ Chinese Acad Sci, Beijing 100049, Peoples R China
推荐引用方式 GB/T 7714	Sun, Sheng,Zhang, Zengqi,Pan, Quyang,et al. Staleness-Controlled Asynchronous Federated Learning: Accuracy and Efficiency Tradeoff[J]. IEEE TRANSACTIONS ON MOBILE COMPUTING,2024,23(12):12621-12634.
APA	Sun, Sheng.,Zhang, Zengqi.,Pan, Quyang.,Liu, Min.,Wang, Yuwei.,...&Wu, Zhiyuan.(2024).Staleness-Controlled Asynchronous Federated Learning: Accuracy and Efficiency Tradeoff.IEEE TRANSACTIONS ON MOBILE COMPUTING,23(12),12621-12634.
MLA	Sun, Sheng,et al."Staleness-Controlled Asynchronous Federated Learning: Accuracy and Efficiency Tradeoff".IEEE TRANSACTIONS ON MOBILE COMPUTING 23.12(2024):12621-12634.