An Efficient Deep Learning Accelerator Architecture for Compressed Video Analysis

doi:10.1109/TCAD.2021.3120076

	An Efficient Deep Learning Accelerator Architecture for Compressed Video Analysis
	Wang, Yongchen 1,2; Wang, Ying 1,2; Li, Huawei 1,2; Li, Xiaowei 1,2
	2022-09-01
发表期刊	IEEE TRANSACTIONS ON COMPUTER-AIDED DESIGN OF INTEGRATED CIRCUITS AND SYSTEMS
ISSN	0278-0070
卷号	41 期号:9 页码:2808-2820
摘要	Previous neural network accelerators tailored to video analysis only accept data of RGB/YUV domain, requiring decompressing the video that are often compressed before transmitted from the edge sensors. A compressed video processing accelerator can alleviate the decoding overhead, and gain performance speedup by operating on more compact input data. This work proposes a novel deep learning accelerator architecture, Alchemist, which is able to predict results directly from the compressed video bitstream instead of reconstructing the full RGB images. By utilizing the metadata of motion vector and critical blocks extracted from bitstreams, Alchemist contributes to a remarkable performance speedup of 5x with negligible accuracy loss. Nevertheless, we still find that the original compressed video coded by standard algorithms such as H.264 is not suitable to be directly manipulated, due to diverse compressed structures. Although obviating the requirement to recover all RGB frames, the accelerator must parse the entire compressed video bitstream to locate reference frames and extract useful metadata. If we combine the video codec with the proposed compressed video analysis, additional optimizations can be obtained. Therefore, to cope with the mismatch between current video coding algorithms, such as H.264 and neural network-based video analysis, we propose a specialized coding strategy to generate compressed video bitstreams more suitable for transmission and analysis, which further simplifies the decoding stage of video analysis and is capable of achieving significant storage reduction.
关键词	Streaming media Neural networks Image coding Decoding Metadata Deep learning Hardware Neural network acceleration specialized accelerator video analysis
DOI	10.1109/TCAD.2021.3120076
收录类别	SCI
语种	英语
资助项目	National Key Research and Development Program of China[2020YFB1600201] ; National Natural Science Foundation of China (NSFC)[61874124] ; National Natural Science Foundation of China (NSFC)[2090024] ; National Natural Science Foundation of China (NSFC)[61876173]
WOS研究方向	Computer Science ; Engineering
WOS类目	Computer Science, Hardware & Architecture ; Computer Science, Interdisciplinary Applications ; Engineering, Electrical & Electronic
WOS记录号	WOS:000842062100007
出版者	IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC
引用统计	被引频次：1[WOS] [WOS记录] [WOS相关记录]
文献类型	期刊论文
条目标识符	http://119.78.100.204/handle/2XEOYT63/19473
专题	中国科学院计算技术研究所期刊论文_英文
通讯作者	Wang, Ying
作者单位	1.Chinese Acad Sci, Inst Comp Technol, State Key Lab Comp Architecture, Beijing 100190, Peoples R China 2.Univ Chinese Acad Sci, Beijing 100190, Peoples R China
推荐引用方式 GB/T 7714	Wang, Yongchen,Wang, Ying,Li, Huawei,et al. An Efficient Deep Learning Accelerator Architecture for Compressed Video Analysis[J]. IEEE TRANSACTIONS ON COMPUTER-AIDED DESIGN OF INTEGRATED CIRCUITS AND SYSTEMS,2022,41(9):2808-2820.
APA	Wang, Yongchen,Wang, Ying,Li, Huawei,&Li, Xiaowei.(2022).An Efficient Deep Learning Accelerator Architecture for Compressed Video Analysis.IEEE TRANSACTIONS ON COMPUTER-AIDED DESIGN OF INTEGRATED CIRCUITS AND SYSTEMS,41(9),2808-2820.
MLA	Wang, Yongchen,et al."An Efficient Deep Learning Accelerator Architecture for Compressed Video Analysis".IEEE TRANSACTIONS ON COMPUTER-AIDED DESIGN OF INTEGRATED CIRCUITS AND SYSTEMS 41.9(2022):2808-2820.