Dadu-SV: Accelerate Stereo Vision Processing on NPU

doi:10.1109/LES.2022.3162859

	Dadu-SV: Accelerate Stereo Vision Processing on NPU
	Min, Feng 1; Wang, Ying 2; Xu, Haobo 1; Huang, Junpei 3; Wang, Yujie 1; Zou, Xingqi 1; Lu, Meixuan 1; Han, Yinhe
	2022-12-01
发表期刊	IEEE EMBEDDED SYSTEMS LETTERS
ISSN	1943-0663
卷号	14 期号:4 页码:191-194
摘要	Binocular vision and neural networks (CNNs) are widely seen in modern intelligent vision processing systems, such as robotics, autonomous vehicles, and AR gadgets. However, both the classic semiglobal matching (SGM) and deep CNNs entail substantial computing resource to reach the performance goal. Traditional embedded CPU/graphic processor unit (GPU) cannot simultaneously meet the processing speed and energy requirement, while the specialized circuits dedicated to SGM and CNN processing, respectively, will take considerable hardware and development costs. However, as the popularity of deep learning, neural processing units (NPUs) become prevalent in many embedded and edge devices, which possess high throughput computing power to deal with the matrix operations involved by neural networks. In this work, we attempt to take advantage of the neural processing architectures integrated in SoC chips to accelerate the SGM process, so that the hardware resources will be better utilized instead of investing more resources to create specialized SGM components. Thereby, this letter first deploys SGM on NPU by converting the incompatible operations into the neural-computing flow, and a configurable neural processing element is proposed to flexibly support various vector operation sequences. Then, a hybrid dataflow scheduler and the corresponding hardware modification are introduced to accelerate the cost processing, improving hardware utilization and on-chip memory footprint and access. Our solution runs at 45 fps for an image size of $640\times 480$ , with 128 disparity levels. The speed-energy efficiency is $52\times $ better than the GPU (Jetson TX1) solution with negligible additional hardware overhead and accuracy loss.
关键词	Hardware acceleration neural computing neural processing unit (NPU) semiglobal matching (SGM) stereo vision
DOI	10.1109/LES.2022.3162859
收录类别	SCI
语种	英语
资助项目	National Natural Science Foundation of China (NSFC)[62025404] ; National Natural Science Foundation of China (NSFC)[61834006] ; National Natural Science Foundation of China (NSFC)[61874124] ; Strategic Priority Research Program of Chinese Academy of Sciences[XDC05030100] ; Strategic Priority Research Program of Chinese Academy of Sciences[XDB4400000] ; Strategic Priority Research Program of Chinese Academy of Sciences[XDPB12]
WOS研究方向	Computer Science ; Engineering
WOS类目	Computer Science, Hardware & Architecture ; Computer Science, Software Engineering ; Engineering, Electrical & Electronic
WOS记录号	WOS:000890850400008
出版者	IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC
引用统计	被引频次：2[WOS] [WOS记录] [WOS相关记录]
文献类型	期刊论文
条目标识符	http://119.78.100.204/handle/2XEOYT63/20274
专题	中国科学院计算技术研究所期刊论文
通讯作者	Wang, Ying; Xu, Haobo
作者单位	1.Chinese Acad Sci, Inst Comp Technol, Res Ctr Intelligent Comp Syst, Beijing 100045, Peoples R China 2.Chinese Acad Sci, Inst Comp Technol, State Key Lab Comp Architecture, Beijing 100190, Peoples R China 3.Univ Sci & Technol China, Sch Microelect, Hefei 230026, Anhui, Peoples R China
推荐引用方式 GB/T 7714	Min, Feng,Wang, Ying,Xu, Haobo,et al. Dadu-SV: Accelerate Stereo Vision Processing on NPU[J]. IEEE EMBEDDED SYSTEMS LETTERS,2022,14(4):191-194.
APA	Min, Feng.,Wang, Ying.,Xu, Haobo.,Huang, Junpei.,Wang, Yujie.,...&Han, Yinhe.(2022).Dadu-SV: Accelerate Stereo Vision Processing on NPU.IEEE EMBEDDED SYSTEMS LETTERS,14(4),191-194.
MLA	Min, Feng,et al."Dadu-SV: Accelerate Stereo Vision Processing on NPU".IEEE EMBEDDED SYSTEMS LETTERS 14.4(2022):191-194.