Stereoscopic Image Retargeting Based on Deep Convolutional Neural Network

doi:10.1109/TCSVT.2021.3054062

	Stereoscopic Image Retargeting Based on Deep Convolutional Neural Network
	Fan, Xiaoting 1; Lei, Jianjun 1; Liang, Jie 2; Fang, Yuming 3; Ling, Nam 4; Huang, Qingming 5,6
	2021-12-01
发表期刊	IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS FOR VIDEO TECHNOLOGY
ISSN	1051-8215
卷号	31 期号:12 页码:4759-4770
摘要	Stereoscopic image retargeting aims at converting stereoscopic images to the target resolution adaptively. Different from 2D image retargeting, stereoscopic image retargeting needs to preserve both the shape structure of salient objects and depth consistency of 3D scenes. In this paper, we present a stereoscopic image retargeting method based on deep convolutional neural network to obtain high-quality retargeted images with both object shape preservation and scene depth preservation. First, a cross-attention extraction mechanism is constructed to generate attention map, which contains the valuable attention features of the left and right images and the common attention features between them. Second, since the disparity map can provide accurate depth information of objects in 3D scenes, a disparity-assisted 3D significance map generation module is utilized to further preserve the valuable depth information of stereoscopic images. Finally, in order to predict the retargeted stereoscopic images accurately, an image consistency loss is developed to preserve the geometric structure of salient objects, and a disparity consistency loss is introduced to eliminate depth distortions. Experimental results demonstrate that the proposed deep convolutional neural network can provide favorable stereoscopic image retargeting results.
关键词	Stereo image processing Three-dimensional displays Two dimensional displays Feature extraction Distortion Visualization Shape Stereoscopic image image retargeting cross-attention disparity consistency
DOI	10.1109/TCSVT.2021.3054062
收录类别	SCI
语种	英语
资助项目	National Natural Science Foundation of China[61931014] ; National Natural Science Foundation of China[61620106009] ; National Natural Science Foundation of China[U1636214] ; National Key Research and Development Program of China[2018YFE0203900] ; Natural Science Foundation of Tianjin[18ZXZNGX00110] ; Natural Science Foundation of Tianjin[18JCJQJC45800] ; China Scholarship Council[201906250188]
WOS研究方向	Engineering
WOS类目	Engineering, Electrical & Electronic
WOS记录号	WOS:000725812500020
出版者	IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC
引用统计	被引频次：6[WOS] [WOS记录] [WOS相关记录]
文献类型	期刊论文
条目标识符	http://119.78.100.204/handle/2XEOYT63/18147
专题	中国科学院计算技术研究所期刊论文_英文
通讯作者	Lei, Jianjun
作者单位	1.Tianjin Univ, Sch Elect & Informat Engn, Tianjin 300072, Peoples R China 2.Simon Fraser Univ, Sch Engn Sci, Burnaby, BC V5A 1S6, Canada 3.Jiangxi Univ Finance & Econ, Sch Informat Technol, Nanchang 330032, Jiangxi, Peoples R China 4.Santa Clara Univ, Dept Comp Sci & Engn, Santa Clara, CA 95053 USA 5.Univ Chinese Acad Sci, Sch Comp Sci & Technol, Beijing 101408, Peoples R China 6.Chinese Acad Sci, Inst Comp Technol, Key Lab Intelligent Informat Proc, Beijing 100190, Peoples R China
推荐引用方式 GB/T 7714	Fan, Xiaoting,Lei, Jianjun,Liang, Jie,et al. Stereoscopic Image Retargeting Based on Deep Convolutional Neural Network[J]. IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS FOR VIDEO TECHNOLOGY,2021,31(12):4759-4770.
APA	Fan, Xiaoting,Lei, Jianjun,Liang, Jie,Fang, Yuming,Ling, Nam,&Huang, Qingming.(2021).Stereoscopic Image Retargeting Based on Deep Convolutional Neural Network.IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS FOR VIDEO TECHNOLOGY,31(12),4759-4770.
MLA	Fan, Xiaoting,et al."Stereoscopic Image Retargeting Based on Deep Convolutional Neural Network".IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS FOR VIDEO TECHNOLOGY 31.12(2021):4759-4770.