IVP: An Intelligent Video Processing Architecture for Video Streaming

doi:10.1109/TC.2022.3155950

	IVP: An Intelligent Video Processing Architecture for Video Streaming
	Gao, Chengsi 1,2; Wang, Ying 3; Han, Yinhe 3; Chen, Weiwei 1,2; Zhang, Lei 1
	2023
发表期刊	IEEE TRANSACTIONS ON COMPUTERS
ISSN	0018-9340
卷号	72 期号:1 页码:264-277
摘要	Recently, video processing tasks, such as video enhancement and analysis, have received increasing attention from both academics and industries. However, the current video processing procedure on edge decouples the decoding phase and the subsequent video processing tasks, missing the opportunity to accelerate the procedure by orchestrating video decoding and enhancement stages. Thus, we propose an intelligent video processing workflow and architecture(IVP) for cloud-edge video streaming. For edge devices that receive compressed videos, IVP can perform direct DNN-based video enhancement, e.g., super-resolution and frame-interpolation. By leveraging the metadata motion vectors and residuals extracted from the encoded video, our architecture will significantly eliminate unnecessary frame pixels being processed by the DNNs and improve execution efficiency. The proposed IVP and workflow are proved to reduce up to 90% of the processing latency while producing accurate and high-quality videos. Furthermore, we observe a significant portion of similar optical flow in time domain of continuous videos, which can be used to reduce the computation overhead. Thus, to utilize such temporal similarity of optical flow, the proposed IVP is upgraded to be capable of reusing previous computation results, which further improves energy efficiency of the whole system.
关键词	Video enhancement compressed video DNN approximate computing optical flow accelerator
DOI	10.1109/TC.2022.3155950
收录类别	SCI
语种	英语
资助项目	National Natural Science Foundation of China[62090024] ; National Natural Science Foundation of China[61874124] ; Strategic Priority Research Program of Chinese Academy of Sciences[XDC05030201] ; Zhejiang Lab[2021PC0AC01]
WOS研究方向	Computer Science ; Engineering
WOS类目	Computer Science, Hardware & Architecture ; Engineering, Electrical & Electronic
WOS记录号	WOS:000899952600021
出版者	IEEE COMPUTER SOC
引用统计	被引频次：4[WOS] [WOS记录] [WOS相关记录]
文献类型	期刊论文
条目标识符	http://119.78.100.204/handle/2XEOYT63/20155
专题	中国科学院计算技术研究所期刊论文
通讯作者	Wang, Ying
作者单位	1.Chinese Acad Sci, Inst Comp Technol, Beijing 100190, Peoples R China 2.Univ Chinese Acad Sci, Beijing 100049, Peoples R China 3.Chinese Acad Sci, Inst Comp Technol, State Key Lab Comp Architecture, Beijing 100190, Peoples R China
推荐引用方式 GB/T 7714	Gao, Chengsi,Wang, Ying,Han, Yinhe,et al. IVP: An Intelligent Video Processing Architecture for Video Streaming[J]. IEEE TRANSACTIONS ON COMPUTERS,2023,72(1):264-277.
APA	Gao, Chengsi,Wang, Ying,Han, Yinhe,Chen, Weiwei,&Zhang, Lei.(2023).IVP: An Intelligent Video Processing Architecture for Video Streaming.IEEE TRANSACTIONS ON COMPUTERS,72(1),264-277.
MLA	Gao, Chengsi,et al."IVP: An Intelligent Video Processing Architecture for Video Streaming".IEEE TRANSACTIONS ON COMPUTERS 72.1(2023):264-277.