CIR-Net: Cross-Modality Interaction and Refinement for RGB-D Salient Object Detection

doi:10.1109/TIP.2022.3216198

	CIR-Net: Cross-Modality Interaction and Refinement for RGB-D Salient Object Detection
	Cong, Runmin 1,2; Lin, Qinwei 1,2; Zhang, Chen 1,2; Li, Chongyi 3; Cao, Xiaochun 4; Huang, Qingming 5,6,7; Zhao, Yao 1,2
	2022
发表期刊	IEEE TRANSACTIONS ON IMAGE PROCESSING
ISSN	1057-7149
卷号	31 页码:6800-6815
摘要	Focusing on the issue of how to effectively capture and utilize cross-modality information in RGB-D salient object detection (SOD) task, we present a convolutional neural network (CNN) model, named CIR-Net, based on the novel cross-modality interaction and refinement. For the cross-modality interaction, 1) a progressive attention guided integration unit is proposed to sufficiently integrate RGB-D feature representations in the encoder stage, and 2) a convergence aggregation structure is proposed, which flows the RGB and depth decoding features into the corresponding RGB-D decoding streams via an importance gated fusion unit in the decoder stage. For the cross-modality refinement, we insert a refinement middleware structure between the encoder and the decoder, in which the RGB, depth, and RGB-D encoder features are further refined by successively using a self-modality attention refinement unit and a cross-modality weighting refinement unit. At last, with the gradually refined features, we predict the saliency map in the decoder stage. Extensive experiments on six popular RGB-D SOD benchmarks demonstrate that our network outperforms the state-of-the-art saliency detectors both qualitatively and quantitatively. The code and results can be found from the link of https://rmcong.github.io/proj_CIRNet.html.
关键词	Decoding Task analysis Periodic structures Middleware Logic gates Electronic mail Object detection Salient object detection RGB-D images cross-modality attention cross-modality interaction
DOI	10.1109/TIP.2022.3216198
收录类别	SCI
语种	英语
资助项目	National Key Research and Development Program of China[2021ZD0112100] ; Beijing Nova Program[Z201100006820016] ; National Natural Science Foundation of China[62002014] ; National Natural Science Foundation of China[U1936212] ; National Natural Science Foundation of China[62120106009] ; National Natural Science Foundation of China[U21B2038] ; National Natural Science Foundation of China[61931008] ; Beijing Natural Science Foundation[4222013] ; China Association for Science and Technology[2020QNRC001] ; CAAI-Huawei MindSpore Open Fund ; Fundamental Research Funds for the Central Universities[2022JBMC002]
WOS研究方向	Computer Science ; Engineering
WOS类目	Computer Science, Artificial Intelligence ; Engineering, Electrical & Electronic
WOS记录号	WOS:000875886800009
出版者	IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC
引用统计	被引频次：51[WOS] [WOS记录] [WOS相关记录]
文献类型	期刊论文
条目标识符	http://119.78.100.204/handle/2XEOYT63/19930
专题	中国科学院计算技术研究所期刊论文
通讯作者	Zhang, Chen; Li, Chongyi
作者单位	1.Beijing Jiaotong Univ, Inst Informat Sci, Beijing 100044, Peoples R China 2.Beijing Key Lab Adv Informat Sci & Network Techno, Beijing 100044, Peoples R China 3.Nanyang Technol Univ, Sch Comp Sci & Engn, Singapore 639798, Singapore 4.Sun Yat Sen Univ, Sch Cyber Sci & Technol, Shenzhen Campus, Shenzhen 518107, Peoples R China 5.Univ Chinese Acad Sci, Sch Comp Sci & Technol, Beijing 101408, Peoples R China 6.Chinese Acad Sci, Inst Comp Technol, Key Lab Intelligent Informat Proc, Beijing 100190, Peoples R China 7.Peng Cheng Lab, Shenzhen 518055, Peoples R China
推荐引用方式 GB/T 7714	Cong, Runmin,Lin, Qinwei,Zhang, Chen,et al. CIR-Net: Cross-Modality Interaction and Refinement for RGB-D Salient Object Detection[J]. IEEE TRANSACTIONS ON IMAGE PROCESSING,2022,31:6800-6815.
APA	Cong, Runmin.,Lin, Qinwei.,Zhang, Chen.,Li, Chongyi.,Cao, Xiaochun.,...&Zhao, Yao.(2022).CIR-Net: Cross-Modality Interaction and Refinement for RGB-D Salient Object Detection.IEEE TRANSACTIONS ON IMAGE PROCESSING,31,6800-6815.
MLA	Cong, Runmin,et al."CIR-Net: Cross-Modality Interaction and Refinement for RGB-D Salient Object Detection".IEEE TRANSACTIONS ON IMAGE PROCESSING 31(2022):6800-6815.