Accelerating Convolutional Neural Networks by Exploiting the Sparsity of Output Activation

doi:10.1109/TPDS.2023.3324934

	Accelerating Convolutional Neural Networks by Exploiting the Sparsity of Output Activation
	Fan, Zhihua 1,2; Li, Wenming 1,2; Wang, Zhen 1,2; Liu, Tianyu 1,2; Wu, Haibin 1,2; Liu, Yanhuan 1,2; Wu, Meng 1,2; Wu, Xinxin 1; Ye, Xiaochun 1; Fan, Dongrui 1,2; Sun, Ninghui 1,2; An, Xuejun 1,2
	2023-12-01
发表期刊	IEEE TRANSACTIONS ON PARALLEL AND DISTRIBUTED SYSTEMS
ISSN	1045-9219
卷号	34 期号:12 页码:3253-3265
摘要	Deep Convolutional Neural Networks (CNNs) are the most widely used family of machine learning methods that have had a transformative effect on a wide range of applications. Previous studies have made great breakthroughs in accelerating CNNs, but they only target on the input sparsity of activation and weight, thus do not eliminate the unnecessary computations due to the fact that more zeros in the output results are not directly caused by the zero-valued positions of the input data. In this paper, we take advantage of the output activation sparsity to reduce the execution time and energy consumption of CNNs. First, we propose an effective prediction method that leverages the output activation sparsity. Our method first predicts the output activation polarity of convolutional layers based on the singular value decomposition (SVD) approach. Then, it uses the predicted negative value to skip invalid computations. Second, an effective accelerator is designed to take advantage of sparsity to achieve CNN inference acceleration. Each PE is equipped with a prediction unit and a non-zero value detection unit to remove invalid computation blocks. And an instruction bypass technique is proposed which further exploits the sparsity of the weights. The efficient dataflow graph mapping approach and pipeline execution ensure high computational resource utilization. Experiments show that our approach achieves up to 1.63x speedup and 55.30% energy reduction compared with dense networks with a slight loss of accuracy. Compared with Eyeriss, our accelerator achieves on average 1.31 x performance improvement and 54% energy reduction. Our accelerator also achieves a similar performance to SnaPEA, but with a better energy efficiency.
关键词	Accelerator output activation prediction sparse convolutional neural network
DOI	10.1109/TPDS.2023.3324934
收录类别	SCI
语种	英语
资助项目	National Key R&D Program of China[2022YFB4501404] ; Beijing Nova Program[2022079] ; CAS Project for Young Scientists in Basic Research[YSBR- 029] ; CAS Project for Youth Innovation Promotion Association and Open Research Projects of Zhejiang Lab[2022PB0AB01]
WOS研究方向	Computer Science ; Engineering
WOS类目	Computer Science, Theory & Methods ; Engineering, Electrical & Electronic
WOS记录号	WOS:001097049800002
出版者	IEEE COMPUTER SOC
引用统计	被引频次：1[WOS] [WOS记录] [WOS相关记录]
文献类型	期刊论文
条目标识符	http://119.78.100.204/handle/2XEOYT63/38094
专题	中国科学院计算技术研究所期刊论文_英文
通讯作者	Li, Wenming
作者单位	1.Chinese Acad Sci, Inst Comp Technol, State Key Lab Processor, Beijing 100045, Peoples R China 2.Univ Chinese Acad Sci, Sch Comp & Control Engn, Beijing 101408, Peoples R China
推荐引用方式 GB/T 7714	Fan, Zhihua,Li, Wenming,Wang, Zhen,et al. Accelerating Convolutional Neural Networks by Exploiting the Sparsity of Output Activation[J]. IEEE TRANSACTIONS ON PARALLEL AND DISTRIBUTED SYSTEMS,2023,34(12):3253-3265.
APA	Fan, Zhihua.,Li, Wenming.,Wang, Zhen.,Liu, Tianyu.,Wu, Haibin.,...&An, Xuejun.(2023).Accelerating Convolutional Neural Networks by Exploiting the Sparsity of Output Activation.IEEE TRANSACTIONS ON PARALLEL AND DISTRIBUTED SYSTEMS,34(12),3253-3265.
MLA	Fan, Zhihua,et al."Accelerating Convolutional Neural Networks by Exploiting the Sparsity of Output Activation".IEEE TRANSACTIONS ON PARALLEL AND DISTRIBUTED SYSTEMS 34.12(2023):3253-3265.