FTT-NAS: Discovering Fault-tolerant Convolutional Neural Architecture

doi:10.1145/3460288

	FTT-NAS: Discovering Fault-tolerant Convolutional Neural Architecture
	Ning, Xuefei 1; Ge, Guangjun 1; Li, Wenshuo 1; Zhu, Zhenhua 1; Zheng, Yin 2; Chen, Xiaoming 3; Gao, Zhen 4; Wang, Yu 1; Yang, Huazhong 1
	2021-11-01
发表期刊	ACM TRANSACTIONS ON DESIGN AUTOMATION OF ELECTRONIC SYSTEMS
ISSN	1084-4309
卷号	26 期号:6 页码:24
摘要	With the fast evolvement of embedded deep-learning computing systems, applications powered by deep learning are moving from the cloud to the edge. When deploying neural networks (NNs) onto the devices under complex environments, there are various types of possible faults: soft errors caused by cosmic radiation and radioactive impurities, voltage instability, aging, temperature variations, malicious attackers, and so on. Thus, the safety risk of deploying NNs is now drawing much attention. In this article, after the analysis of the possible faults in various types of NN accelerators, we formalize and implement various fault models from the algorithmic perspective. We propose Fault-Tolerant Neural Architecture Search (FT-NAS) to automatically discover convolutional neural network (CNN) architectures that are reliable to various faults in nowadays devices. Then, we incorporate fault-tolerant training (FTT) in the search process to achieve better results, which is referred to as FTT-NAS. Experiments on CIFAR-10 show that the discovered architectures outperform other manually designed baseline architectures significantly, with comparable or fewer floating-point operations (FLOPs) and parameters. Specifically, with the same fault settings, F-FTT-Net discovered under the feature fault model achieves an accuracy of 86.2% (VS. 68.1% achieved by MobileNet-V2), and W-FTT-Net discovered under the weight fault model achieves an accuracy of 69.6% (VS. 60.8% achieved by ResNet-18). By inspecting the discovered architectures, we find that the operation primitives, the weight quantization range, the capacity of the model, and the connection pattern have influences on the fault resilience capability of NN models.
关键词	Neural architecture search fault tolerance neural networks
DOI	10.1145/3460288
收录类别	SCI
语种	英语
资助项目	National Natural Science Foundation of China[U19B2019] ; National Natural Science Foundation of China[61832007] ; National Natural Science Foundation of China[61621091] ; National Key R&D Program of China[2017YFA02077600] ; Beijing National Research Center for Information Science and Technology (BNRist) ; Beijing Innovation Center for Future Chips ; Tsinghua University[TT2020-01] ; Toyota Joint Research Center for AI Technology of Automated Vehicle[TT2020-01] ; Beijing Academy of Artificial Intelligence
WOS研究方向	Computer Science
WOS类目	Computer Science, Hardware & Architecture ; Computer Science, Software Engineering
WOS记录号	WOS:000756208000004
出版者	ASSOC COMPUTING MACHINERY
引用统计	被引频次：11[WOS] [WOS记录] [WOS相关记录]
文献类型	期刊论文
条目标识符	http://119.78.100.204/handle/2XEOYT63/19007
专题	中国科学院计算技术研究所期刊论文_英文
通讯作者	Ning, Xuefei
作者单位	1.Tsinghua Univ, Dept Elect Engn, Rohm Bldg, Beijing 100084, Peoples R China 2.Tencent, Weixin Grp, Beijing 100080, Peoples R China 3.Chinese Acad Sci, Inst Comp Technol, State Key Lab Comp Architecture, Beijing 100190, Peoples R China 4.Tianjin Univ, Sch Elect & Informat Engn, Tianjin 300072, Peoples R China
推荐引用方式 GB/T 7714	Ning, Xuefei,Ge, Guangjun,Li, Wenshuo,et al. FTT-NAS: Discovering Fault-tolerant Convolutional Neural Architecture[J]. ACM TRANSACTIONS ON DESIGN AUTOMATION OF ELECTRONIC SYSTEMS,2021,26(6):24.
APA	Ning, Xuefei.,Ge, Guangjun.,Li, Wenshuo.,Zhu, Zhenhua.,Zheng, Yin.,...&Yang, Huazhong.(2021).FTT-NAS: Discovering Fault-tolerant Convolutional Neural Architecture.ACM TRANSACTIONS ON DESIGN AUTOMATION OF ELECTRONIC SYSTEMS,26(6),24.
MLA	Ning, Xuefei,et al."FTT-NAS: Discovering Fault-tolerant Convolutional Neural Architecture".ACM TRANSACTIONS ON DESIGN AUTOMATION OF ELECTRONIC SYSTEMS 26.6(2021):24.