Learning Coupled Convolutional Networks Fusion for Video Saliency Prediction

doi:10.1109/TCSVT.2018.2870954

	Learning Coupled Convolutional Networks Fusion for Video Saliency Prediction
	Wu, Zhe 1,2; Su, Li 1,2; Huang, Qingming 1,2,3
	2019-10-01
发表期刊	IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS FOR VIDEO TECHNOLOGY
ISSN	1051-8215
卷号	29 期号:10 页码:2960-2971
摘要	Visual saliency provides important information for understanding scenes in many computer vision tasks. The existing video saliency algorithms mainly focus on predicting spatial and temporal saliency maps. However, these maps are simply fused without considering the complex dynamic scenes in videos. To overcome this drawback, we propose a deep convolutional fusion framework for video saliency prediction. The proposed model, which is based on coupled fully convolutional networks (FCNs), effectively encodes the spatiotemporal information by integrating spatial and temporal features. We demonstrate that this information is helpful for accurately fusing the spatial and temporal saliency maps according to changes in video scenes. In particular, we gradually design three different deep fusion architectures to investigate how to better utilize the spatiotemporal information. Moreover, we propose a reasonable sampling strategy for selecting suitable training sets for the coupled FCNs. Through extensive experiments, we demonstrate that our model outperforms the state-of-the-art algorithms on four public video saliency data sets.
关键词	Video saliency feature integration fully convolutional network
DOI	10.1109/TCSVT.2018.2870954
收录类别	SCI
语种	英语
资助项目	National Natural Sciences Foundation of China[61472389] ; National Natural Sciences Foundation of China[61650202] ; National Natural Sciences Foundation of China[61332016] ; National Natural Sciences Foundation of China[61620106009] ; National Natural Sciences Foundation of China[U1636214] ; Key Research Program of Frontier Sciences[CAS: QYZDJ-SSW-SYS013]
WOS研究方向	Engineering
WOS类目	Engineering, Electrical & Electronic
WOS记录号	WOS:000489749900008
出版者	IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC
引用统计	被引频次：12[WOS] [WOS记录] [WOS相关记录]
文献类型	期刊论文
条目标识符	http://119.78.100.204/handle/2XEOYT63/4597
专题	中国科学院计算技术研究所期刊论文_英文
通讯作者	Su, Li
作者单位	1.UCAS, Sch Comp & Control Engn, Beijing 101408, Peoples R China 2.UCAS, Key Lab Big Data Min & Knowledge Management, Beijing 101408, Peoples R China 3.Chinese Acad Sci, Key Lab Intelligent Informat Proc, Inst Comp Technol, Beijing 100190, Peoples R China
推荐引用方式 GB/T 7714	Wu, Zhe,Su, Li,Huang, Qingming. Learning Coupled Convolutional Networks Fusion for Video Saliency Prediction[J]. IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS FOR VIDEO TECHNOLOGY,2019,29(10):2960-2971.
APA	Wu, Zhe,Su, Li,&Huang, Qingming.(2019).Learning Coupled Convolutional Networks Fusion for Video Saliency Prediction.IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS FOR VIDEO TECHNOLOGY,29(10),2960-2971.
MLA	Wu, Zhe,et al."Learning Coupled Convolutional Networks Fusion for Video Saliency Prediction".IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS FOR VIDEO TECHNOLOGY 29.10(2019):2960-2971.