An Automatic Neural Network Architecture-and-Quantization Joint Optimization Framework for Efficient Model Inference

doi:10.1109/TCAD.2023.3339438

	An Automatic Neural Network Architecture-and-Quantization Joint Optimization Framework for Efficient Model Inference
	Liu, Lian 1,2; Wang, Ying 1,2; Zhao, Xiandong 3; Chen, Weiwei ; Li, Huawei 1,2; Li, Xiaowei 1,2; Han, Yinhe 1,2
	2024-05-01
发表期刊	IEEE TRANSACTIONS ON COMPUTER-AIDED DESIGN OF INTEGRATED CIRCUITS AND SYSTEMS
ISSN	0278-0070
卷号	43 期号:5 页码:1497-1510
摘要	Efficient deep learning models, especially optimized for edge devices, benefit from low inference latency to efficient energy consumption. Two classical techniques for efficient model inference are lightweight neural architecture search (NAS), which automatically designs compact network models, and quantization, which reduces the bit-precision of neural network models. As a consequence, joint design for both neural architecture and quantization precision settings is becoming increasingly popular. There are three main aspects that affect the performance of the joint optimization between neural architecture and quantization: 1) quantization precision selection (QPS); 2) quantization-aware training (QAT); and 3) NAS. However, existing works focus on at most twofold of these aspects, and result in secondary performance. To this end, we proposed a novel automatic optimization framework, DAQU, that allows jointly searching for Pareto-optimal neural architecture and quantization precision combination among more than $10<^>{47}$ quantized subnet models. To overcome the instability of the conventional automatic optimization framework, DAQU incorporates a warm-up strategy to reduce the accuracy gap among different neural architectures, and a precision-transfer training approach to maintain flexibility among different quantization precision settings. Our experiments show that the quantized lightweight neural networks generated by DAQU consistently outperform state-of-the-art NAS and quantization joint optimization methods.
关键词	Optimization Quantization (signal) Computer architecture Training Computational modeling Integrated circuit modeling Convergence Automatic joint optimization efficient model inference network quantization neural architecture search (NAS)
DOI	10.1109/TCAD.2023.3339438
收录类别	SCI
语种	英语
资助项目	National Natural Science Foundation of China (NSFC)
WOS研究方向	Computer Science ; Engineering
WOS类目	Computer Science, Hardware & Architecture ; Computer Science, Interdisciplinary Applications ; Engineering, Electrical & Electronic
WOS记录号	WOS:001225897600014
出版者	IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC
引用统计
文献类型	期刊论文
条目标识符	http://119.78.100.204/handle/2XEOYT63/40075
专题	中国科学院计算技术研究所期刊论文_英文
通讯作者	Wang, Ying; Li, Huawei
作者单位	1.Chinese Acad Sci, Inst Comp Technol, State Key Lab Processors, Beijing 100190, Peoples R China 2.Univ Chinese Acad Sci, Dept Comp Sci, Beijing 100190, Peoples R China 3.Chinese Acad Sci, Inst Comp Technol, Beijing 100190, Peoples R China
推荐引用方式 GB/T 7714	Liu, Lian,Wang, Ying,Zhao, Xiandong,et al. An Automatic Neural Network Architecture-and-Quantization Joint Optimization Framework for Efficient Model Inference[J]. IEEE TRANSACTIONS ON COMPUTER-AIDED DESIGN OF INTEGRATED CIRCUITS AND SYSTEMS,2024,43(5):1497-1510.
APA	Liu, Lian.,Wang, Ying.,Zhao, Xiandong.,Chen, Weiwei.,Li, Huawei.,...&Han, Yinhe.(2024).An Automatic Neural Network Architecture-and-Quantization Joint Optimization Framework for Efficient Model Inference.IEEE TRANSACTIONS ON COMPUTER-AIDED DESIGN OF INTEGRATED CIRCUITS AND SYSTEMS,43(5),1497-1510.
MLA	Liu, Lian,et al."An Automatic Neural Network Architecture-and-Quantization Joint Optimization Framework for Efficient Model Inference".IEEE TRANSACTIONS ON COMPUTER-AIDED DESIGN OF INTEGRATED CIRCUITS AND SYSTEMS 43.5(2024):1497-1510.