Model Aggregation Method for Data Parallelism in Distributed Real-Time Machine Learning of Smart Sensing Equipment

doi:10.1109/ACCESS.2019.2955547

	Model Aggregation Method for Data Parallelism in Distributed Real-Time Machine Learning of Smart Sensing Equipment
	Fan, Yuchen 1,2; Zhang, Jilin 1,2,3; Zhao, Nailiang 1,2; Ren, Yongjian 1,2; Wan, Jian 1,2,4; Zhou, Li 1,2; Shen, Zhongyu 1,2; Wang, Jue 5; Zhang, Juncong 6; Wei, Zhenguo 6
	2019
发表期刊	IEEE ACCESS
ISSN	2169-3536
卷号	7 页码:172065-172073
摘要	In distributed real-time machine learning of smart sensing equipment, training speed and training accuracy are two hard-to-choose trade-off performance measures directly influenced by the design of distributed machine learning algorithms. And it will influence effort of smart sensing equipment directly. We take the model aggregation method of distributed machine learning as a starting point. Due to the loss of accuracy caused by the direct averaging of the parameter average method, we developed the loss function weight reorder stochastic gradient descent method (LR-SGD). LR-SGD uses the loss function value to determine the weight of the work nodes when aggregating the model parameters, and it improves the performance of the parameter average method for nonconvex problems. As shown in the experiment results, our algorithm can improve the training accuracy by a maximum of approximately 0.57% for the Bulk Synchronous Parallel (BSP) model and approximately 6.30% for the Stale Synchronous Parallel (SSP) model.
关键词	Distributed machine learning stochastic gradient descent model aggregation method smart sensing equipment
DOI	10.1109/ACCESS.2019.2955547
收录类别	SCI
语种	英语
资助项目	National Key Technology Research and Development Program[2018YFB0204001] ; National Natural Science Foundation of China[61672200] ; National Natural Science Foundation of China[61572163] ; Key Technology Research and Development Program of the Zhejiang Province[2019C01059] ; Zhejiang Natural Science Funds[LY17F020029] ; Zhejiang Natural Science Funds[LY16F020018] ; State Key Laboratory of Computer Architecture Project[CARCH201712] ; Hangzhou Dianzi University Postgraduate Research Innovation Fund Program[CXJJ2018052]
WOS研究方向	Computer Science ; Engineering ; Telecommunications
WOS类目	Computer Science, Information Systems ; Engineering, Electrical & Electronic ; Telecommunications
WOS记录号	WOS:000509374200057
出版者	IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC
引用统计	被引频次：5[WOS] [WOS记录] [WOS相关记录]
文献类型	期刊论文
条目标识符	http://119.78.100.204/handle/2XEOYT63/14736
专题	中国科学院计算技术研究所期刊论文_英文
通讯作者	Zhao, Nailiang; Ren, Yongjian
作者单位	1.Hangzhou Dianzi Univ, Sch Comp Sci & Technol, Hangzhou 310018, Peoples R China 2.Minist Educ, Key Lab Complex Syst Modeling & Simulat, Hangzhou 310018, Peoples R China 3.Chinese Acad Sci, Inst Comp Technol, State Key Lab Comp Architecture, Beijing 100190, Peoples R China 4.Zhejiang Univ Sci & Technol, Hangzhou 310023, Peoples R China 5.Chinese Acad Sci, Comp Network Informat Ctr, Beijing 100190, Peoples R China 6.Zhejiang Dawning Informat Technol Co Ltd, Hangzhou 310051, Zhejiang, Peoples R China
推荐引用方式 GB/T 7714	Fan, Yuchen,Zhang, Jilin,Zhao, Nailiang,et al. Model Aggregation Method for Data Parallelism in Distributed Real-Time Machine Learning of Smart Sensing Equipment[J]. IEEE ACCESS,2019,7:172065-172073.
APA	Fan, Yuchen.,Zhang, Jilin.,Zhao, Nailiang.,Ren, Yongjian.,Wan, Jian.,...&Wei, Zhenguo.(2019).Model Aggregation Method for Data Parallelism in Distributed Real-Time Machine Learning of Smart Sensing Equipment.IEEE ACCESS,7,172065-172073.
MLA	Fan, Yuchen,et al."Model Aggregation Method for Data Parallelism in Distributed Real-Time Machine Learning of Smart Sensing Equipment".IEEE ACCESS 7(2019):172065-172073.