Efficient and Fast High-Performance Library Generation for Deep Learning Accelerators

doi:10.1109/TC.2024.3475575

	Efficient and Fast High-Performance Library Generation for Deep Learning Accelerators
	Bi, Jun 1; Wen, Yuanbo 1; Li, Xiaqing 1; Zhao, Yongwei 1; Guo, Yuxuan 1,3,4; Zhou, Enshuai 1,3,4; Hu, Xing 1,2; Du, Zidong 1,2; Li, Ling 5; Chen, Huaping 3; Chen, Tianshi 4; Guo, Qi 1
	2025
发表期刊	IEEE TRANSACTIONS ON COMPUTERS
ISSN	0018-9340
卷号	74 期号:1 页码:155-169
摘要	The widespread adoption of deep learning accelerators (DLAs) underscores their pivotal role in improving the performance and energy efficiency of neural networks. To fully leverage the capabilities of these accelerators, exploration-based library generation approaches have been widely used to substantially reduce software development overhead. However, these approaches have been challenged by issues related to sub-optimal optimization results and excessive optimization overheads. In this paper, we propose Heron to generate high-performance libraries of DLAs in an efficient and fast way. The key is automatically enforcing massive constraints through the entire program generation process and guiding the exploration with an accurate pre-trained cost model. Heron represents the search space as a constrained satisfaction problem (CSP) and explores the space via evolving the CSPs. Thus, the sophisticated constraints of the search space are strictly preserved during the entire exploration process. The exploration algorithm has the flexibility to engage in space exploration using either online-trained models or pre-trained models. Experimental results demonstrate that Heron averagely achieves 2.71$\times$x speedup over three state-of-the-art automatic generation approaches. Also, compared to vendor-provided hand-tuned libraries, Heron achieves a 2.00$\times$x speedup on average. When employing a pre-trained model, Heron achieves 11.6$\times$x compilation time speedup, incurring a minor impact on execution time.
关键词	Optimization Space exploration Schedules Libraries Biological cells Deep learning Costs Computers Search problems Tensors Code generation compiler optimization tensor computation
DOI	10.1109/TC.2024.3475575
收录类别	SCI
语种	英语
资助项目	National Key R&D Program of China[2021ZD0110102] ; NSF of China[U22A2028] ; NSF of China[62302483] ; NSF of China[62372436] ; NSF of China[62102398] ; NSF of China[62222214] ; CAS Project for Young Scientists in Basic Research[YSBR-029] ; Youth Innovation Promotion Association CAS
WOS研究方向	Computer Science ; Engineering
WOS类目	Computer Science, Hardware & Architecture ; Engineering, Electrical & Electronic
WOS记录号	WOS:001377244200018
出版者	IEEE COMPUTER SOC
引用统计
文献类型	期刊论文
条目标识符	http://119.78.100.204/handle/2XEOYT63/41069
专题	中国科学院计算技术研究所期刊论文_英文
通讯作者	Wen, Yuanbo
作者单位	1.Chinese Acad Sci, Inst Comp Technol, State Key Lab Processors, Beijing 100864, Peoples R China 2.Shanghai Innovat Ctr Processor Technol SH, Shanghai 200000, Peoples R China 3.Univ Sci & Technol China, Hefei 230052, Anhui, Peoples R China 4.Cambricon Technol, Beijing 100191, Peoples R China 5.Chinese Acad Sci, Inst Software, Intelligent Software Res Ctr, Beijing 100190, Peoples R China
推荐引用方式 GB/T 7714	Bi, Jun,Wen, Yuanbo,Li, Xiaqing,et al. Efficient and Fast High-Performance Library Generation for Deep Learning Accelerators[J]. IEEE TRANSACTIONS ON COMPUTERS,2025,74(1):155-169.
APA	Bi, Jun.,Wen, Yuanbo.,Li, Xiaqing.,Zhao, Yongwei.,Guo, Yuxuan.,...&Guo, Qi.(2025).Efficient and Fast High-Performance Library Generation for Deep Learning Accelerators.IEEE TRANSACTIONS ON COMPUTERS,74(1),155-169.
MLA	Bi, Jun,et al."Efficient and Fast High-Performance Library Generation for Deep Learning Accelerators".IEEE TRANSACTIONS ON COMPUTERS 74.1(2025):155-169.