Does Thermal Really Always Matter for RGB-T Salient Object Detection?

doi:10.1109/TMM.2022.3216476

	Does Thermal Really Always Matter for RGB-T Salient Object Detection?
	Cong, Runmin 1,2,3; Zhang, Kepu 1,2; Zhang, Chen 1,2; Zheng, Feng 4,5; Zhao, Yao 1,2; Huang, Qingming 6,7,8; Kwong, Sam 3,9
	2023
发表期刊	IEEE TRANSACTIONS ON MULTIMEDIA
ISSN	1520-9210
卷号	25 页码:6971-6982
摘要	In recent years, RGB-T salient object detection (SOD) has attracted continuous attention, which makes it possible to identify salient objects in environments such as low light by introducing thermal image. However, most of the existing RGB-T SOD models focus on how to perform cross-modality feature fusion, ignoring whether thermal image is really always matter in SOD task. Starting from the definition and nature of this task, this paper rethinks the connotation of thermal modality, and proposes a network named TNet to solve the RGB-T SOD task. In this paper, we introduce a global illumination estimation module to predict the global illuminance score of the image, so as to regulate the role played by the two modalities. In addition, considering the role of thermal modality, we set up different cross-modality interaction mechanisms in the encoding phase and the decoding phase. On the one hand, we introduce a semantic constraint provider to enrich the semantics of thermal images in the encoding phase, which makes thermal modality more suitable for the SOD task. On the other hand, we introduce a two-stage localization and complementation module in the decoding phase to transfer object localization cue and internal integrity cue in thermal features to the RGB modality. Extensive experiments on three datasets show that the proposed TNet achieves competitive performance compared with 20 state-of-the-art methods.
关键词	Task analysis Decoding Semantics Object detection Location awareness Lighting Feature extraction RGB-T images salient object detection global illumination estimation semantic constraint provider localization and complementation
DOI	10.1109/TMM.2022.3216476
收录类别	SCI
语种	英语
资助项目	National Key R&D Program of China[2021ZD0112100] ; Beijing Nova Program[Z201100006820016] ; National Natural Science Foundation of China[62002014] ; National Natural Science Foundation of China[U1936212] ; National Natural Science Foundation of China[62120106009] ; National Natural Science Foundation of China[62236008] ; National Natural Science Foundation of China[U21B2038] ; National Natural Science Foundation of China[61972188] ; National Natural Science Foundation of China[62122035] ; Beijing Natural Science Foundation[4222013] ; Hong Kong Innovation and Technology Commission (InnoHK Project CIMDA) ; Hong Kong GRF-RGC General Research Fund[11209819 (CityU 9042816)] ; Hong Kong GRF-RGC General Research Fund[11203820 (CityU 9042598)] ; Young Elite Scientist Sponsorship Program by the China Association for Science and Technology[2020QNRC001] ; CAAI-Huawei MindSpore Open Fund ; Dr Cong's Project ; Fundamental Research Funds for the Central Universities[2022JBMC002]
WOS研究方向	Computer Science ; Telecommunications
WOS类目	Computer Science, Information Systems ; Computer Science, Software Engineering ; Telecommunications
WOS记录号	WOS:001102654000019
出版者	IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC
引用统计	被引频次：35[WOS] [WOS记录] [WOS相关记录]
文献类型	期刊论文
条目标识符	http://119.78.100.204/handle/2XEOYT63/38078
专题	中国科学院计算技术研究所期刊论文_英文
通讯作者	Zheng, Feng
作者单位	1.Beijing Jiaotong Univ, Inst Informat Sci, Beijing 100044, Peoples R China 2.Network Technol, Beijing Key Lab Adv Informat Sci, Beijing 100044, Peoples R China 3.City Univ Hong Kong, Dept Comp Sci, Hong Kong, Peoples R China 4.Southern Univ Sci & Technol, Dept Comp Sci & Technol, Shenzhen 518055, Peoples R China 5.Res Inst Trustworthy Autonomous Syst, Shenzhen 518055, Peoples R China 6.Univ Chinese Acad Sci, Sch Comp Sci & Technol, Beijing 101408, Peoples R China 7.Chinese Acad Sci, Inst Comp Technol, Key Lab Intelligent Informat Proc, Beijing 100190, Peoples R China 8.Peng Cheng Lab, Shenzhen 518055, Peoples R China 9.City Univ Hong Kong, Shenzhen Res Inst, Shenzhen 51800, Peoples R China
推荐引用方式 GB/T 7714	Cong, Runmin,Zhang, Kepu,Zhang, Chen,et al. Does Thermal Really Always Matter for RGB-T Salient Object Detection?[J]. IEEE TRANSACTIONS ON MULTIMEDIA,2023,25:6971-6982.
APA	Cong, Runmin.,Zhang, Kepu.,Zhang, Chen.,Zheng, Feng.,Zhao, Yao.,...&Kwong, Sam.(2023).Does Thermal Really Always Matter for RGB-T Salient Object Detection?.IEEE TRANSACTIONS ON MULTIMEDIA,25,6971-6982.
MLA	Cong, Runmin,et al."Does Thermal Really Always Matter for RGB-T Salient Object Detection?".IEEE TRANSACTIONS ON MULTIMEDIA 25(2023):6971-6982.