An Application-oblivious Memory Scheduling System for DNN Accelerators

doi:10.1145/3535355

	An Application-oblivious Memory Scheduling System for DNN Accelerators
	Li, Jiansong 1; Wang, Xueying 2,3; Chen, Xiaobing 2,3; Li, Guangli 2,3; Dong, Xiao 4; Zhao, Peng 5; Yu, Xianzhi 5; Yang, Yongxin 2,3; Cao, Wei 2,3; Liu, Lei 2,3; Feng, Xiaobing 2,3
	2022-12-01
发表期刊	ACM TRANSACTIONS ON ARCHITECTURE AND CODE OPTIMIZATION
ISSN	1544-3566
卷号	19 期号:4 页码:26
摘要	Deep Neural Networks (DNNs) tend to go deeper and wider, which poses a significant challenge to the training of DNNs, due to the limited memory capacity of DNN accelerators. Existing solutions for memory-efficient DNN training are densely coupled with the application features of DNN workloads, e.g., layer structures or computational graphs of DNNs are necessary for these solutions. This would result in weak versatility for DNNs with sophisticated layer structures or complicated computation graphs. These schemes usually need to be re-implemented or re-adapted due to the new layer structures or the unusual operators in the computational graphs introduced by these DNNs. In this article, we review the memory pressure issues of DNN training from the perspective of runtime systems and model the memory access behaviors of DNN workloads. We identify the iterative, regularity, and extremalization properties of memory access patterns for DNN workloads. Based on these observations, we propose AppObMem, an application-oblivious memory scheduling system. AppObMem automatically traces the memory behaviors of DNN workloads and schedules the memory swapping to reduce the memory pressure of the device accelerators without the perception of high-level information of layer structures or computation graphs. Evaluations on a variety ofDNNmodels showthat, AppObMem obtains 40-60% memory savings with acceptable performance loss. AppObMem is also competitive with other open sourced SOTA schemes.
关键词	Deep learning memory scheduling runtime system DNN accelerators
DOI	10.1145/3535355
收录类别	SCI
语种	英语
资助项目	National Natural Science Foundation of China[61872043]
WOS研究方向	Computer Science
WOS类目	Computer Science, Hardware & Architecture ; Computer Science, Theory & Methods
WOS记录号	WOS:000893255000001
出版者	ASSOC COMPUTING MACHINERY
引用统计	被引频次：1[WOS] [WOS记录] [WOS相关记录]
文献类型	期刊论文
条目标识符	http://119.78.100.204/handle/2XEOYT63/20223
专题	中国科学院计算技术研究所期刊论文
通讯作者	Li, Guangli
作者单位	1.Huawei Galois Lab, Beijing, Peoples R China 2.Chinese Acad Sci, State Key Lab Comp Architecture, Inst Comp Technol, 6 Kexueyuan South Rd, Beijing 100190, Peoples R China 3.Univ Chinese Acad Sci, Sch Comp Sci & Technol, 19 A Yuquan Rd, Beijing 100049, Peoples R China 4.NVIDIA Corp, Shanghai, Peoples R China 5.Huawei 2012 Lab, Beijing, Peoples R China
推荐引用方式 GB/T 7714	Li, Jiansong,Wang, Xueying,Chen, Xiaobing,et al. An Application-oblivious Memory Scheduling System for DNN Accelerators[J]. ACM TRANSACTIONS ON ARCHITECTURE AND CODE OPTIMIZATION,2022,19(4):26.
APA	Li, Jiansong.,Wang, Xueying.,Chen, Xiaobing.,Li, Guangli.,Dong, Xiao.,...&Feng, Xiaobing.(2022).An Application-oblivious Memory Scheduling System for DNN Accelerators.ACM TRANSACTIONS ON ARCHITECTURE AND CODE OPTIMIZATION,19(4),26.
MLA	Li, Jiansong,et al."An Application-oblivious Memory Scheduling System for DNN Accelerators".ACM TRANSACTIONS ON ARCHITECTURE AND CODE OPTIMIZATION 19.4(2022):26.