VTensor: Using Virtual Tensors to Build a Layout-Oblivious AI Programming Framework

doi:10.1007/s11390-022-1457-6

	VTensor: Using Virtual Tensors to Build a Layout-Oblivious AI Programming Framework
	Yu, Feng 1,2; Zhao, Jia-Cheng 1,2; Cui, Hui-Min 1,2; Feng, Xiao-Bing 1,2; Xue, Jingling 3
	2023-09-01
发表期刊	JOURNAL OF COMPUTER SCIENCE AND TECHNOLOGY
ISSN	1000-9000
卷号	38 期号:5 页码:1074-1097
摘要	Tensors are a popular programming interface for developing artificial intelligence (AI) algorithms. Layout refers to the order of placing tensor data in the memory and will affect performance by affecting data locality; therefore the deep neural network library has a convention on the layout. Since AI applications can use arbitrary layouts, and existing AI systems do not provide programming abstractions to shield the layout conventions of libraries, operator developers need to write a lot of layout-related code, which reduces the efficiency of integrating new libraries or developing new operators. Furthermore, the developer assigns the layout conversion operation to the internal operator to deal with the uncertainty of the input layout, thus losing the opportunity for layout optimization. Based on the idea of polymorphism, we propose a layout-agnostic virtual tensor programming interface, namely the VTensor framework, which enables developers to write new operators without caring about the underlying physical layout of tensors. In addition, the VTensor framework performs global layout inference at runtime to transparently resolve the required layout of virtual tensors, and runtime layout-oriented optimizations to globally minimize the number of layout transformation operations. Experimental results demonstrate that with VTensor, developers can avoid writing layout-dependent code. Compared with TensorFlow, for the 16 operations used in 12 popular networks, VTensor can reduce the lines of code (LOC) of writing a new operation by 47.82% on average, and improve the overall performance by 18.65% on average.
关键词	artificial intelligence (AI) programming layout-oblivious tensor processing
DOI	10.1007/s11390-022-1457-6
收录类别	SCI
语种	英语
资助项目	National Key Research and Development Program of China[2021ZD0110101] ; National Natural Science Foundation of China[62090024] ; National Natural Science Foundation of China[61872043] ; National Natural Science Foundation of China[61802368] ; Australian Research Council[DP180104069] ; Australian Research Council[DP210102409]
WOS研究方向	Computer Science
WOS类目	Computer Science, Hardware & Architecture ; Computer Science, Software Engineering
WOS记录号	WOS:001114345700008
出版者	SPRINGER SINGAPORE PTE LTD
引用统计
文献类型	期刊论文
条目标识符	http://119.78.100.204/handle/2XEOYT63/38460
专题	中国科学院计算技术研究所期刊论文_英文
通讯作者	Cui, Hui-Min
作者单位	1.Chinese Acad Sci, Inst Comp Technol, Beijing 100190, Peoples R China 2.Univ Chinese Acad Sci, Sch Comp Sci & Technol, Beijing 100080, Peoples R China 3.Univ New South Wales, Sch Comp Sci & Engn, Sydney 1466, Australia
推荐引用方式 GB/T 7714	Yu, Feng,Zhao, Jia-Cheng,Cui, Hui-Min,et al. VTensor: Using Virtual Tensors to Build a Layout-Oblivious AI Programming Framework[J]. JOURNAL OF COMPUTER SCIENCE AND TECHNOLOGY,2023,38(5):1074-1097.
APA	Yu, Feng,Zhao, Jia-Cheng,Cui, Hui-Min,Feng, Xiao-Bing,&Xue, Jingling.(2023).VTensor: Using Virtual Tensors to Build a Layout-Oblivious AI Programming Framework.JOURNAL OF COMPUTER SCIENCE AND TECHNOLOGY,38(5),1074-1097.
MLA	Yu, Feng,et al."VTensor: Using Virtual Tensors to Build a Layout-Oblivious AI Programming Framework".JOURNAL OF COMPUTER SCIENCE AND TECHNOLOGY 38.5(2023):1074-1097.