Evaluating and analyzing the energy efficiency of CNN inference on high-performance GPU

doi:10.1002/cpe.6064

	Evaluating and analyzing the energy efficiency of CNN inference on high-performance GPU
	Yao, Chunrong 1; Liu, Wantao 2; Tang, Weiqing 1,3; Guo, Jinrong 2; Hu, Songlin 2; Lu, Yijun 4; Jiang, Wei 5
	2020-10-21
发表期刊	CONCURRENCY AND COMPUTATION-PRACTICE & EXPERIENCE
ISSN	1532-0626
页码	26
摘要	Convolutional neural network (CNN) inference usually runs on high-performance graphic processing units (GPUs). Since GPU is a high power consumption unit, that makes the energy consumption increases sharply due to the deep learning tasks. The energy efficiency of CNN inference is not only related to the software and hardware configurations, but also closely related to the application requirements of inference tasks. However, it is not clear on GPUs at present. In this paper, we conduct a comprehensive study on the model-level and layer-level energy efficiency of popular CNN models. The results point out several opportunities for further optimization. We also analyze the parameter settings (i.e., batch size, dynamic voltage and frequency scaling) and propose a revenue model to allow an optimal trade-off between energy efficiency and latency. Compared with the default settings, the optimal settings can improve revenue by up to 15.31x. We obtain the following main findings: (i) GPUs do not exploit the parallelism from the model depth and small convolution kernels, resulting in low energy efficiency. (ii) Convolutional layers are the most energy-consuming CNN layers. However, due to the cache, the power consumption of all layers is relatively balanced. (iii) The energy efficiency of TensorRT is 1.53xthan that of TensorFlow.
关键词	CNNs energy efficiency high-performance GPU inference
DOI	10.1002/cpe.6064
收录类别	SCI
语种	英语
资助项目	National Key Research and Development Program of China[2017YFB1010000]
WOS研究方向	Computer Science
WOS类目	Computer Science, Software Engineering ; Computer Science, Theory & Methods
WOS记录号	WOS:000580529000001
出版者	WILEY
引用统计	被引频次：16[WOS] [WOS记录] [WOS相关记录]
文献类型	期刊论文
条目标识符	http://119.78.100.204/handle/2XEOYT63/15726
专题	中国科学院计算技术研究所期刊论文_英文
通讯作者	Liu, Wantao
作者单位	1.Nanjing Univ Sci & Technol, Sch Comp Sci & Engn, Nanjing, Peoples R China 2.Chinese Acad Sci, Inst Informat Engn, Beijing 100093, Peoples R China 3.Chinese Acad Sci, Inst Comp Technol, Beijing, Peoples R China 4.Alibaba Cloud Comp Co Ltd, Hangzhou, Peoples R China 5.State Grid Corp China, Dept Energy Internet, Beijing, Peoples R China
推荐引用方式 GB/T 7714	Yao, Chunrong,Liu, Wantao,Tang, Weiqing,et al. Evaluating and analyzing the energy efficiency of CNN inference on high-performance GPU[J]. CONCURRENCY AND COMPUTATION-PRACTICE & EXPERIENCE,2020:26.
APA	Yao, Chunrong.,Liu, Wantao.,Tang, Weiqing.,Guo, Jinrong.,Hu, Songlin.,...&Jiang, Wei.(2020).Evaluating and analyzing the energy efficiency of CNN inference on high-performance GPU.CONCURRENCY AND COMPUTATION-PRACTICE & EXPERIENCE,26.
MLA	Yao, Chunrong,et al."Evaluating and analyzing the energy efficiency of CNN inference on high-performance GPU".CONCURRENCY AND COMPUTATION-PRACTICE & EXPERIENCE (2020):26.