Deep Intra Prediction by Jointly Exploiting Local and Non-Local Similarities

doi:10.1109/TCSVT.2022.3220434

	Deep Intra Prediction by Jointly Exploiting Local and Non-Local Similarities
	Lei, Meng 1; Zhang, Jiaqi 2; Wang, Shiqi 3,4; Wang, Shanshe 5,6,7; Ma, Siwei 5,6,7
	2023-05-01
发表期刊	IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS FOR VIDEO TECHNOLOGY
ISSN	1051-8215
卷号	33 期号:5 页码:2396-2409
摘要	Intra prediction, which aims to remove the redundancies within a frame, has shown promising performance by simply projecting and interpolating samples along multiple angular directions. Recently, with numerous approaches devoted to learning nonlinear predictors with deep neural networks (DNN) based on local correlations, much less work has been dedicated to exploring non-local self-similarities in intra prediction. In this paper, we propose a unified prediction model that exploits both local and non-local correlations for intra prediction. The proposed model not only supports the nonlinear prediction using local reference samples as input, but also aggregates useful non-local information from a large reconstructed region with a Patch-level Non-local Attention Network (PNA-Net). More specifically, PNA-Net incorporates template matching with attention mechanism in feature domain to obtain the responses of all non-local features to the content to be predicted, leading to the prediction produced with weighted non-local patches. Finally, the predictions in the local and non-local manners are blended adaptively with a trainable network, ensuring the capability to handle a variety of contents. Experimental results on Versatile Video Coding (VVC) software VTM-11.0 show that the proposed model achieves on average 4.69% bit rate savings for natural scene sequences, and 4.24% bit rate savings for screen content sequences under the all intra configuration.
关键词	Correlation Predictive models Deep learning Video coding Image reconstruction Encoding Convolutional neural networks Intra prediction attention mechanism template matching non-local operation
DOI	10.1109/TCSVT.2022.3220434
收录类别	SCI
语种	英语
资助项目	National Natural Science Foundation of China[62031013] ; National Natural Science Foundation of China[62025101] ; National Natural Science Foundation of China[62088102] ; High Performance Computing Platform of Peking University
WOS研究方向	Engineering
WOS类目	Engineering, Electrical & Electronic
WOS记录号	WOS:000982426900029
出版者	IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC
引用统计
文献类型	期刊论文
条目标识符	http://119.78.100.204/handle/2XEOYT63/21436
专题	中国科学院计算技术研究所期刊论文_英文
通讯作者	Ma, Siwei
作者单位	1.Peking Univ, Natl Engn Res Ctr Visual Technol, Sch Comp Sci, Beijing 100871, Peoples R China 2.Chinese Acad Sci, Inst Comp Technol, Key Lab Intelligent Informat Proc, Beijing 100190, Peoples R China 3.Peng Cheng Lab, Shenzhen 518066, Peoples R China 4.City Univ Hong Kong, Dept Comp Sci, Hong Kong, Peoples R China 5.Peking Univ, Natl Engn Res Ctr Visual Technol, Sch Comp Sci, Beijing 100871, Peoples R China 6.Peking Univ, Informat Technol Res & Dev Innovat Ctr, Shaoxing 312000, Peoples R China 7.Peng Cheng Lab, Shenzhen 518066, Peoples R China
推荐引用方式 GB/T 7714	Lei, Meng,Zhang, Jiaqi,Wang, Shiqi,et al. Deep Intra Prediction by Jointly Exploiting Local and Non-Local Similarities[J]. IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS FOR VIDEO TECHNOLOGY,2023,33(5):2396-2409.
APA	Lei, Meng,Zhang, Jiaqi,Wang, Shiqi,Wang, Shanshe,&Ma, Siwei.(2023).Deep Intra Prediction by Jointly Exploiting Local and Non-Local Similarities.IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS FOR VIDEO TECHNOLOGY,33(5),2396-2409.
MLA	Lei, Meng,et al."Deep Intra Prediction by Jointly Exploiting Local and Non-Local Similarities".IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS FOR VIDEO TECHNOLOGY 33.5(2023):2396-2409.