Statistical Modeling of Soft Error Influence on Neural Networks

doi:10.1109/TCAD.2023.3266405

	Statistical Modeling of Soft Error Influence on Neural Networks
	Huang, Haitong 1,2; Xue, Xinghua 1,2; Liu, Cheng 1,2; Wang, Ying 1,2; Luo, Tao 3; Cheng, Long 4; Li, Huawei 1; Li, Xiaowei 1,2
	2023-11-01
发表期刊	IEEE TRANSACTIONS ON COMPUTER-AIDED DESIGN OF INTEGRATED CIRCUITS AND SYSTEMS
ISSN	0278-0070
卷号	42 期号:11 页码:4152-4163
摘要	Soft errors in large VLSI circuits have a significant impact on computing- and memory-intensive neural network (NN) processing. Understanding the influence of soft errors on NNs is critical to protect against soft errors for reliable NN processing. Prior work mainly relies on fault simulation to analyze the influence of soft errors on NN processing. They are accurate but usually specific to limited configurations of errors and NN models due to the prohibitively slow simulation speed especially for large NN models and datasets. With the observation that the influence of soft errors propagates across a large number of neurons and accumulates as well, we propose to characterize the soft error-induced data disturbance on each neuron with a normal distribution model using the central limit theorem and develop a series of statistical models to analyze the behavior of NN models under soft errors in general. The statistical models reveal not only the correlation between soft errors and the accuracy of NN models but also how NN parameters, such as quantization and architecture affect the reliability of NNs. The proposed models are compared with fault simulations and verified comprehensively. In addition, we observe that the statistical models that characterize the soft error influence can also be utilized to predict fault simulation results in many cases and we explore the use of the proposed statistical models to accelerate fault simulations of NNs. Our experiments show that the proposed accelerated fault simulation provides almost two orders of magnitude speedup with negligible loss of simulation accuracy compared to the baseline fault simulations.
关键词	Fault analysis fault simulation neural network (NN) reliability statistical fault modeling
DOI	10.1109/TCAD.2023.3266405
收录类别	SCI
语种	英语
资助项目	National Natural Science Foundation of China[62174162] ; National Key Research and Development Program of China[2022YFB4500405] ; Singapore Government's Research, Innovation and Enterprise 2020 Plan (Advanced Manufacturing and Engineering Domain)[A1687b0033]
WOS研究方向	Computer Science ; Engineering
WOS类目	Computer Science, Hardware & Architecture ; Computer Science, Interdisciplinary Applications ; Engineering, Electrical & Electronic
WOS记录号	WOS:001098114300051
出版者	IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC
引用统计	被引频次：4[WOS] [WOS记录] [WOS相关记录]
文献类型	期刊论文
条目标识符	http://119.78.100.204/handle/2XEOYT63/38086
专题	中国科学院计算技术研究所期刊论文_英文
通讯作者	Liu, Cheng
作者单位	1.Chinese Acad Sci, Inst Comp Technol, State Key Lab Processors, Beijing 100190, Peoples R China 2.Univ Chinese Acad Sci, Dept Comp Sci, Beijing 100190, Peoples R China 3.ASTAR, Inst High Performance Comp, Singapore 138632, Singapore 4.North China Elect Power Univ, Sch Control & Comp Engn, Beijing 102206, Peoples R China
推荐引用方式 GB/T 7714	Huang, Haitong,Xue, Xinghua,Liu, Cheng,et al. Statistical Modeling of Soft Error Influence on Neural Networks[J]. IEEE TRANSACTIONS ON COMPUTER-AIDED DESIGN OF INTEGRATED CIRCUITS AND SYSTEMS,2023,42(11):4152-4163.
APA	Huang, Haitong.,Xue, Xinghua.,Liu, Cheng.,Wang, Ying.,Luo, Tao.,...&Li, Xiaowei.(2023).Statistical Modeling of Soft Error Influence on Neural Networks.IEEE TRANSACTIONS ON COMPUTER-AIDED DESIGN OF INTEGRATED CIRCUITS AND SYSTEMS,42(11),4152-4163.
MLA	Huang, Haitong,et al."Statistical Modeling of Soft Error Influence on Neural Networks".IEEE TRANSACTIONS ON COMPUTER-AIDED DESIGN OF INTEGRATED CIRCUITS AND SYSTEMS 42.11(2023):4152-4163.