CoAxNN: Optimizing on-device deep learning with conditional approximate neural networks

doi:10.1016/j.sysarc.2023.102978

	CoAxNN: Optimizing on-device deep learning with conditional approximate neural networks
	Li, Guangli 1,2; Ma, Xiu 3,4; Yu, Qiuchu 1,2; Liu, Lei 3,4; Liu, Huaxiao 3,4; Wang, Xueying 1,2
	2023-10-01
发表期刊	JOURNAL OF SYSTEMS ARCHITECTURE
ISSN	1383-7621
卷号	143 页码:14
摘要	While deep neural networks have achieved superior performance in a variety of intelligent applications, the increasing computational complexity makes them difficult to be deployed on resource-constrained devices. To improve the performance of on-device inference, prior studies have explored various approximate strategies, such as neural network pruning, to optimize models based on different principles. However, when combining these approximate strategies, a large parameter space needs to be explored. Meanwhile, different configuration parameters may interfere with each other, damaging the performance optimization effect. In this paper, we propose a novel model optimization framework, CoAxNN, which effectively combines different approximate strategies, to facilitate on-device deep learning via model approximation. Based on the principles of different approximate optimizations, our approach constructs the design space and automatically finds reasonable configurations through genetic algorithm-based design space exploration. By combining the strengths of different approximation methods, CoAxNN enables efficient conditional inference for models at runtime. We evaluate our approach by leveraging state-of-the-art neural networks on a representative intelligent edge platform, Jetson AGX Orin. The experimental results demonstrate the effectiveness of CoAxNN, which achieves up to 1.53x speedup while reducing energy by up to 34.61%, with trivial accuracy loss on CIFAR-10/100 and CINIC-10 datasets.
关键词	On-device deep learning Efficient neural networks Model approximation and optimization
DOI	10.1016/j.sysarc.2023.102978
收录类别	SCI
语种	英语
资助项目	National Key Ramp;D Program of China[2021ZD0110101] ; National Natural Science Foundation of China[62232015] ; National Natural Science Foundation of China[62302479] ; China Postdoctoral Science Foundation[2023M733566] ; CCF-Baidu Open Fund, China
WOS研究方向	Computer Science
WOS类目	Computer Science, Hardware & Architecture ; Computer Science, Software Engineering
WOS记录号	WOS:001071128000001
出版者	ELSEVIER
引用统计
文献类型	期刊论文
条目标识符	http://119.78.100.204/handle/2XEOYT63/21146
专题	中国科学院计算技术研究所期刊论文_英文
通讯作者	Wang, Xueying
作者单位	1.Chinese Acad Sci, Inst Comp Technol, State Key Lab Processors, Beijing, Peoples R China 2.Univ Chinese Acad Sci, Beijing, Peoples R China 3.Jilin Univ, Coll Comp Sci & Technol, Changchun, Peoples R China 4.Jilin Univ, MOE Key Lab Symbol Computat & Knowledge Engn, Changchun, Peoples R China
推荐引用方式 GB/T 7714	Li, Guangli,Ma, Xiu,Yu, Qiuchu,et al. CoAxNN: Optimizing on-device deep learning with conditional approximate neural networks[J]. JOURNAL OF SYSTEMS ARCHITECTURE,2023,143:14.
APA	Li, Guangli,Ma, Xiu,Yu, Qiuchu,Liu, Lei,Liu, Huaxiao,&Wang, Xueying.(2023).CoAxNN: Optimizing on-device deep learning with conditional approximate neural networks.JOURNAL OF SYSTEMS ARCHITECTURE,143,14.
MLA	Li, Guangli,et al."CoAxNN: Optimizing on-device deep learning with conditional approximate neural networks".JOURNAL OF SYSTEMS ARCHITECTURE 143(2023):14.