Space-address decoupled scratchpad memory management for neural network accelerators

doi:10.1002/cpe.6046

	Space-address decoupled scratchpad memory management for neural network accelerators
	Zhang, Zhenxing 1,2,3; Sun, Shiyan 4; Chen, Xunyu 3; Zhi, Tian 2; Guo, Qi 2; Chen, Yunji 2,5
	2020-10-13
发表期刊	CONCURRENCY AND COMPUTATION-PRACTICE & EXPERIENCE
ISSN	1532-0626
页码	13
摘要	Deep neural networks have been demonstrated to be useful in varieties of intelligent tasks, and various specialized NN accelerators have been proposed recently to improve the hardware efficiency, which are typically equipped with software-managed scratchpad memory (SPM) for high performance and energy efficiency. However, traditional SPM management techniques cause memory fragmentation for NN accelerators, and thus lead to low utilization of precious SPM. The main reason is that traditional techniques are originally designed for managingfixed-length registersrather thanvariable-length memory blocks. In this article, we propose a novel SPM management approach for NN accelerators. The basic intuition is that NN computation/memory behaviors are predictable and relatively regular compared with traditional applications, and thus most information can be determined at compile time. In addition, by exploiting the variable-length feature of SPM, we propose to divide the allocation process into two passes: thespace assignmentand theaddress assignmentpass, which are simultaneously (and implicitly) performed in traditional one-pass allocation techniques. Experimental results on the memory requests of a representative NN accelerator demonstrate that the proposed approach can significantly reduce the memory consumption by 30% at most compared with state-of-the-art SPM management techniques, and the memory usage is only 2% larger than that of the theoretical optimal allocation.
关键词	deep neural network memory management scratchpad memory
DOI	10.1002/cpe.6046
收录类别	SCI
语种	英语
资助项目	National Key Research and Development Program of China[2017YFB1003101] ; National Key Research and Development Program of China[2018AAA0103300] ; National Key Research and Development Program of China[2017YFA0700900] ; National Key Research and Development Program of China[2017YFA0700902] ; National Key Research and Development Program of China[2017YFA0700901] ; National Key Research and Development Program of China[2019AAA0103802] ; NSF of China[61532016] ; NSF of China[61732007] ; NSF of China[61672491] ; NSF of China[61732002] ; NSF of China[61925208] ; NSF of China[61702478] ; NSF of China[6196179] ; NSF of China[61976200] ; NSF of China[U19B2019] ; Beijing Natural Science Foundation[JQ18013] ; Beijing New Generation Artificial Intelligence Technology Cultivation Projects[Z181100008918020] ; National Science and Technology Major Project[2018ZX01031102] ; Key Research Projects in Frontier Science of Chinese Academy of Sciences[QYZDB-SSW-JSC001] ; Strategic Priority Research Program of Chinese Academy of Science[XDB32050200] ; Strategic Priority Research Program of Chinese Academy of Science[XDC01020000] ; Strategic Priority Research Program of Chinese Academy of Science[XDC05010300] ; Strategic Priority Research Program of Chinese Academy of Science[XDC08040102] ; Standardization Research Project of Chinese Academy of Sciences[BZ201800001] ; Beijing Academy of Artificial Intelligence (BAAI) ; Beijing Nova Program of Science and Technology[Z191100001119093] ; Guangdong Science and Technology Program[2019B090909005] ; Xplore Prize
WOS研究方向	Computer Science
WOS类目	Computer Science, Software Engineering ; Computer Science, Theory & Methods
WOS记录号	WOS:000579081000001
出版者	WILEY
引用统计
文献类型	期刊论文
条目标识符	http://119.78.100.204/handle/2XEOYT63/15722
专题	中国科学院计算技术研究所期刊论文_英文
通讯作者	Chen, Xunyu
作者单位	1.Univ Sci & Technol China, Sch Comp Sci & Technol, Hefei, Peoples R China 2.Nanjing Univ Aeronaut & Astronaut, Nanjing, Peoples R China 3.Chinese Acad Sci, Inst Comp Technol, SKL Comp Architecture, Beijing, Peoples R China 4.Cambricon Technol, Beijing, Peoples R China 5.Univ Chinese Acad Sci, Sch Comp Sci & Technol, Beijing, Peoples R China
推荐引用方式 GB/T 7714	Zhang, Zhenxing,Sun, Shiyan,Chen, Xunyu,et al. Space-address decoupled scratchpad memory management for neural network accelerators[J]. CONCURRENCY AND COMPUTATION-PRACTICE & EXPERIENCE,2020:13.
APA	Zhang, Zhenxing,Sun, Shiyan,Chen, Xunyu,Zhi, Tian,Guo, Qi,&Chen, Yunji.(2020).Space-address decoupled scratchpad memory management for neural network accelerators.CONCURRENCY AND COMPUTATION-PRACTICE & EXPERIENCE,13.
MLA	Zhang, Zhenxing,et al."Space-address decoupled scratchpad memory management for neural network accelerators".CONCURRENCY AND COMPUTATION-PRACTICE & EXPERIENCE (2020):13.