Multi-label double-layer learning for cross-modal retrieval

doi:10.1016/j.neucom.2017.10.032

	Multi-label double-layer learning for cross-modal retrieval
	He, Jianfeng 1,2; Ma, Bingpeng 1,2; Wang, Shuhui 2; Liu, Yugui 1; Huang, Qingming 1,2
	2018-01-31
发表期刊	NEUROCOMPUTING
ISSN	0925-2312
卷号	275 页码:1893-1902
摘要	This paper proposes a novel method named Multi-label Double-layer Learning (MDLL) for multi-label cross-modal retrieval task. MDLL includes two stages (layers): L2C (Label to Common) and C2L (Common to Label). In the L2C stage, considering that labels can provide semantic information, we take label information as an auxiliary modality and apply a covariance matrix to represent label similarity in multi-label situation. Thus we can maximize the correlation of different modalities and reduce their semantic gap in the L2C stage. In addition, we find that samples with the same semantic labels may have different contents from users' view. According to this problem, in the C2L stage, labels are projected to a latent space learned from features of image and text. By this way, the label latent space are more related to the sample's contents. Then, it is noticed that the samples have same labels but various contents can be decreased. In MDLL, iterative learning of the L2C and C2L stages will improve the discriminative ability greatly and decline the discrepancy between the labels and the contents. To show the effectiveness of MDLL, some experiments are conducted on three multi-label cross-modal retrieval tasks (Pascal Voc 2007, Nus-wide, and LabelMe), on which competitive results are obtained. (C) 2017 Elsevier B.V. All rights reserved.
关键词	Cross-modal retrieval Multi-label Multimedia Partial least squares
DOI	10.1016/j.neucom.2017.10.032
收录类别	SCI
语种	英语
资助项目	National Basic Research Program of China (973 Program)[2015CB351800] ; Natural Science Foundation of China (NSFC)[61332016] ; Natural Science Foundation of China (NSFC)[61572465] ; Natural Science Foundation of China (NSFC)[61672497] ; Natural Science Foundation of China (NSFC)[61620106009] ; Natural Science Foundation of China (NSFC)[U1636214] ; Natural Science Foundation of China (NSFC)[61650202] ; Key Research Program of Frontier Sciences[CAS: QYZDJ-SSW-SYS013]
WOS研究方向	Computer Science
WOS类目	Computer Science, Artificial Intelligence
WOS记录号	WOS:000418370200176
出版者	ELSEVIER SCIENCE BV
引用统计	被引频次：3[WOS] [WOS记录] [WOS相关记录]
文献类型	期刊论文
条目标识符	http://119.78.100.204/handle/2XEOYT63/5542
专题	中国科学院计算技术研究所期刊论文_英文
通讯作者	Ma, Bingpeng
作者单位	1.Univ China Acad Sci, Sch Comp & Control Engn, Beijing, Peoples R China 2.Chinese Acad Sci, Inst Comp Technol, Key Lab Intelligent Informat Proc, Beijing, Peoples R China
推荐引用方式 GB/T 7714	He, Jianfeng,Ma, Bingpeng,Wang, Shuhui,et al. Multi-label double-layer learning for cross-modal retrieval[J]. NEUROCOMPUTING,2018,275:1893-1902.
APA	He, Jianfeng,Ma, Bingpeng,Wang, Shuhui,Liu, Yugui,&Huang, Qingming.(2018).Multi-label double-layer learning for cross-modal retrieval.NEUROCOMPUTING,275,1893-1902.
MLA	He, Jianfeng,et al."Multi-label double-layer learning for cross-modal retrieval".NEUROCOMPUTING 275(2018):1893-1902.