Saliency Prediction Network for 360 degrees Videos

doi:10.1109/JSTSP.2019.2955824

	Saliency Prediction Network for 360 degrees Videos
	Zhang, Youqiang 1,2; Dai, Feng 1; Ma, Yike 1; Li, Hongliang 2; Zhao, Qiang 1; Zhang, Yongdong 3
	2020
发表期刊	IEEE JOURNAL OF SELECTED TOPICS IN SIGNAL PROCESSING
ISSN	1932-4553
卷号	14 期号:1 页码:27-37
摘要	Panoramic videos are becoming more and more easily obtained for common users. Although these videos have 360 field of view, they are usually displayed with perspective views, which needs the saliency informations for viewing angle selection. In this paper, we propose a saliency prediction network for 360 videos. Our network takes video frames and optical flows in cube map format as input, thus it does not suffer from image distorations of panoramic frames. The network is composed of feature encoding module and saliency prediction module. The feature encoding module extracts spatial and temporal features. Then these features are processed by a decoder and bidirectional convolutional LSTM for saliency prediction. To more thoroughly mine the motion information, the temporal stream of feature encoding module accepts optical flows before and after current frame. We also incorporate the global feature of video frames, residual attention and Gaussian priors into the network by considering the viewing behavior of 360 videos, which is useful for performance improvement. To evaluate the performance of our method, we compare it with three state-of-the-art saliency prediction algorithms on two publicly available datasets. The experimental result has shown the effectiveness of our method, which gets the best performance.
关键词	Saliency prediction 360 degrees videos cube map optical flow global feature Gaussian priors residual attention
DOI	10.1109/JSTSP.2019.2955824
收录类别	SCI
语种	英语
资助项目	National Key R&D Program of China[2018YFB0804203] ; National Natural Science Foundation of China[61702479] ; National Natural Science Foundation of China[61771458] ; Science and Technology Innovation 2030[2018AAA0103000] ; Beijing Municipal Natural Science Foundation Cooperation Beijing Education Committee[KZ-201810005002]
WOS研究方向	Engineering
WOS类目	Engineering, Electrical & Electronic
WOS记录号	WOS:000515666100003
出版者	IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC
引用统计	被引频次：9[WOS] [WOS记录] [WOS相关记录]
文献类型	期刊论文
条目标识符	http://119.78.100.204/handle/2XEOYT63/14584
专题	中国科学院计算技术研究所期刊论文_英文
通讯作者	Zhao, Qiang
作者单位	1.Chinese Acad Sci, Inst Comp Technol, Key Lab Intelligent Informat Proc, Beijing 100190, Peoples R China 2.Univ Chinese Acad Sci, Beijing 100049, Peoples R China 3.Univ Sci & Technol China, Sch Informat Sci & Technol, Hefei 230026, Peoples R China
推荐引用方式 GB/T 7714	Zhang, Youqiang,Dai, Feng,Ma, Yike,et al. Saliency Prediction Network for 360 degrees Videos[J]. IEEE JOURNAL OF SELECTED TOPICS IN SIGNAL PROCESSING,2020,14(1):27-37.
APA	Zhang, Youqiang,Dai, Feng,Ma, Yike,Li, Hongliang,Zhao, Qiang,&Zhang, Yongdong.(2020).Saliency Prediction Network for 360 degrees Videos.IEEE JOURNAL OF SELECTED TOPICS IN SIGNAL PROCESSING,14(1),27-37.
MLA	Zhang, Youqiang,et al."Saliency Prediction Network for 360 degrees Videos".IEEE JOURNAL OF SELECTED TOPICS IN SIGNAL PROCESSING 14.1(2020):27-37.